Image processing device and method thereof, imaging element, and imaging device

ABSTRACT

The present disclosure relates to an image processing device and a method thereof, an imaging element, and an imaging device in which a signal value deviation generated through encoding and decoding of an amplified signal group can be restrained. Adaptive processing is executed on an image in which signal amplification has been executed, and the resultant image is encoded. For example, an offset value that is randomly set within a value range that depends on a gain value of the signal amplification executed on the image is added to each pixel value of the image, and then, the resultant image is encoded. The present disclosure is applicable to an image processing device, an image encoding device, an image decoding device, an imaging element, or an imaging device, for example.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International PatentApplication No. PCT/JP2019/005570 filed on Feb. 15, 2019, which claimspriority benefit of Japanese Patent Application No. JP 2018-036212 filedin the Japan Patent Office on Mar. 1, 2018. Each of the above-referencedapplications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an image processing device and amethod thereof, an imaging element, and an imaging device, andparticularly, relates to an image processing device and a methodthereof, an imaging element, and an imaging device by which a signalvalue deviation generated through encoding and decoding of an amplifiedsignal group can be restrained.

BACKGROUND ART

Various methods are conventionally proposed as methods for encoding(compressing) and decoding (decompressing) images. For example, a methodfor encoding (compressing) image data to a fixed length through DPCM(Differential Pulse Code Modulation) of image data sets and throughaddition of refinement data, is proposed (for example, see PTL 1).

CITATION LIST Patent Literature

[PTL 1]

Japanese Patent Laid-Open No. 2014-103543

SUMMARY Technical Problem

However, when a captured image obtained by high-digital gain imaging,which is for amplifying pixel signals by means of an imaging element orthe like, is encoded and decoded by this method, a pixel deviation mayoccur in the decoded image.

The present disclosure has been arrived at in view of these conditions,and can restrain a signal value deviation which is generated throughencoding and decoding of an amplified signal group.

Solution to Problem

An image processing device according to one aspect of the presenttechnology includes an adaptive processing section that executesadaptive image processing of an image in which signal amplification hasbeen executed, and an encoding section that executes simple encoding ofthe image having undergone the adaptive image processing executed by theadaptive processing section.

An image processing method according to the one aspect of the presenttechnology includes executing adaptive image processing of an image inwhich signal amplification has been executed, and executing simpleencoding of the image having undergone the adaptive image processing.

An imaging element according to another aspect of the present technologyincludes an imaging section that captures an image of a subject, anadaptive processing section that executes adaptive image processing ofthe captured image which has been generated by the imaging section andin which signal amplification has been executed, and an encoding sectionthat executes simple encoding of the captured image having undergone theadaptive image processing executed by the adaptive processing section.

An imaging device according to still another aspect of the presenttechnology includes an imaging element including an imaging section thatcaptures an image of a subject, an adaptive processing section thatexecutes adaptive image processing of the captured image which has beengenerated by the imaging section and in which signal amplification hasbeen executed, and an encoding section that generates encoded data byexecuting simple encoding of the captured image having undergone theadaptive image processing executed by the adaptive processing section,and a decoding section that executes simple decoding of the encoded datagenerated by the encoding section.

In the image processing device according to the one aspect of thepresent technology, adaptive image processing is executed on an image inwhich signal amplification has been executed, and simple encoding of theimage having undergone the adaptive image processing is executed.

In the imaging element according to the other aspect of the presenttechnology, adaptive image processing is executed on a captured imagewhich has been generated by capturing an image of a subject and in whichsignal amplification has been executed, and simple encoding of thecaptured image having undergone the adaptive image processing isexecuted.

In the imaging device according to the still other aspect of the presenttechnology, adaptive image processing is executed on a captured imagewhich has been generated by capturing an image of a subject and in whichsignal amplification has been executed, simple encoding of the capturedimage having undergone the adaptive image processing is executed, andsimple decoding of the encoded data thus generated is executed.

Advantageous Effect of Invention

According to the present disclosure, images can be processed. Inparticular, a signal value deviation, which is generated throughencoding and decoding of an amplified signal group, can be restrained.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1A, 1B, and 1C are diagrams depicting a histogram of a capturedimage.

FIG. 2 is a diagram depicting an example of fixed length encoding.

FIG. 3 is a diagram depicting an example of DC deviation.

FIG. 4 is a diagram depicting a list of processing methods adopting thepresent technology.

FIG. 5 is a block diagram depicting a main configuration example of animage processing system for performing a method #1.

FIGS. 6A and 6B are diagrams depicting an example of a change in ahistogram generated as a result of processing.

FIG. 7 is a diagram depicting a main configuration example of a randomoffset adding section.

FIG. 8 is a diagram depicting an example of a syntax for imposing alimitation on the value range of an offset.

FIGS. 9A and 9B are diagrams depicting an example of imposing alimitation on the value range of an offset.

FIG. 10 is a flowchart for explaining an example of the flow of anencoding process based on the method #1.

FIG. 11 is a flowchart for explaining an example of the flow of anoffset addition process.

FIG. 12 is a flowchart for explaining an example of the flow of adecoding process based on the method #1.

FIG. 13 is a block diagram depicting another configuration example of animage processing system that performs the method #1.

FIG. 14 is a block diagram depicting a main configuration example of animage processing system that performs a method #2.

FIG. 15 is a block diagram depicting a main configuration example of asubtraction offset setting section.

FIG. 16 is a diagram depicting an example of a table which is used forselecting an offset.

FIG. 17 is a flowchart for explaining an example of the flow of anencoding process based on the method #2.

FIG. 18 is a flowchart for explaining an example of the flow of anoffset value setting process.

FIG. 19 is a flowchart for explaining an example of the flow of adecoding process based on the method #2.

FIG. 20 is a block diagram depicting another configuration example of asubtraction offset setting section.

FIG. 21 is a flowchart for explaining an example of the flow of anoffset value setting process.

FIG. 22 is a block diagram depicting another configuration example of animage processing system that performs the method #2.

FIG. 23 is a block diagram depicting another configuration example of animage processing system that performs a method #3.

FIG. 24 is a diagram depicting an example of a table which is used forselecting the range of a quantization value.

FIGS. 25A and 25B are diagrams depicting an example of setting the rangeof a quantization value.

FIG. 26 is a diagram depicting a configuration example of encoded data.

FIG. 27 is a flowchart for explaining an example of the flow of anencoding process based on the method #3.

FIG. 28 is a flowchart for explaining an example of the flow of adecoding process based on the method #3.

FIG. 29 is a block diagram depicting another configuration example of animage processing system that performs the method #3.

FIG. 30 is a block diagram depicting a main configuration example of animage processing system that performs a method #4.

FIG. 31 is a flowchart for explaining an example of the flow of anencoding process based on the method #4.

FIG. 32 is a flowchart for explaining an example of the flow of adecoding process based on the method #4.

FIG. 33 is a block diagram depicting another configuration example of animage processing system that performs the method #4.

FIG. 34 is a diagram depicting a main configuration example of animaging element to which the present technology is applied.

FIG. 35 is a diagram depicting a main configuration example of animaging element to which the present technology is applied.

FIG. 36 is a flowchart for explaining an example of the flow of animaging process.

FIG. 37 is a diagram depicting a main configuration example of animaging device to which the present technology is applied.

FIG. 38 is a flowchart for explaining an example of the flow of animaging process.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments for carrying out the present disclosure(hereinafter, referred to as embodiments) will be explained. It is to benoted that the explanations will be given in accordance with thefollowing order.

1. Fixed Length Encoding

2. General Concept (Outline of Methods)

3. First Embodiment (Details of Method #1)

4. Second Embodiment (Details of Method #2)

5. Third Embodiment (Details of Method #3)

6. Fourth Embodiment (Details of Method #4)

7. Fifth Embodiment (Application Example: Imaging Element)

8. Sixth Embodiment (Application Example: Imaging Device)

9. Supplementary Note

1. Fixed Length Encoding

<Support Documents Etc. Supporting Technical Matters and TechnicalTerms>

The scope disclosed by the present technology encompasses not only thedisclosure in the embodiments, but also the disclosures in the followingdocuments which have been publicly known at the time of filing of thepresent application.

PTL 1: (see above)

PTL 2: Japanese Patent Laid-Open No. 2006-303689

PTL 3: US 2011/0292247

PTL 4: US 2012/0219231

That is, the disclosures in the above documents also constitute thegrounds for determining the support requirements.

<High-Digital Gain Imaging>

There is an imaging method called high-digital gain imaging ofmultiplying a captured image by a prescribed gain value in order tocarry out imaging in a dark place, for example. A case in which ahistogram in FIG. 1A is obtained from a captured image (e.g., a capturedimage that is obtained with a lens case left unremoved) that is obtainedby capturing a black image, for example, is assumed. It is to be notedthat, in the histogram illustrated in FIG. 1A, the horizontal axisindicates a pixel value while the vertical axis indicates a frequency(the number of pixels).

When this captured image is multiplied by a digital gain that isincreased eight-fold in order to enhance the sensitivity, the differenceamong the pixel values of respective pixels is increased eight-fold.Therefore, the histogram of this image is widened, as illustrated inFIG. 1B. That is, the histogram which is dense in FIG. 1A is changed tobe sparse in FIG. 1B in which the values are dispersed to multiples of8, that is, 48, 56, 64, 72, 80, etc., for example.

<Generation of DC Deviation Through Encoding and Decoding>

Meanwhile, various methods have been conventionally proposed as methodsfor encoding (compressing) and decoding (decompressing) images. Forexample, a method for executing fixed length encoding (compression) ofimage data through DPCM (Differential Pulse Code Modulation) among imagedata sets and through addition of refinement data has been proposed, asdisclosed in PTL 1 to 4.

However, if a captured image obtained through the aforementioned digitalgain imaging is encoded and decoded by this method, a histogram such asthat illustrated in FIG. 1C, for example, is obtained for the decodedimage. That is, errors in pixel values are generated only on the +direction side. Thus, there is a possibility that a deviation of anaverage pixel value in the decoded image (also referred to as DCdeviation) is generated.

<Principle of Generation of DC Deviation>

Generation of DC deviation will be more specifically explained. First,the aforementioned fixed length encoding will be explained. FIG. 2 is aschematic diagram depicting image data including pixel blocks including16 pixels (Pixels 1 to 16). Squares of each pixel in FIG. 2 representbits of a pixel value. Each square illustrated on the top represents anMSB (Most Significant Bit), and each square illustrated on the bottomrepresents an LSB (Least Significant Bit). That is, each pixel value is10-bit data.

The aforementioned fixed length encoding is executed for each of theblocks. First, each pixel value in a block is quantized, and aprescribed number of bits (lower bits) from the LSB are deleted. Thatis, only bits which are represented by white squares in FIG. 2 remain.Next, the difference of the quantized pixel value from that in the nextpixel is calculated (DPCM is executed). The obtained differential value(DPCM residual) is encoded data.

More specifically, the pixel data in the block of FIG. 2 is processed inorder from the left side to the right side in FIG. 2 , for example. PCM(Pulse Code Modulation) encoding is executed on higher 7 bits (sevenbits from the MSB) in pixel data to be processed first (the leftmostcolumn in FIG. 2 ). That is, while being in an uncompressed state,higher 7 bits in pixel data to be processed first is outputted asencoded data. Then, DPCM (Differential Pulse Code Modulation) encodingis executed on pixel data to be processed second or later. That is, forhigher 7 bits in the second or later pixel data from the left in FIG. 2, subtraction of higher 7 bits of the preceding (left side in FIG. 2 )pixel data is executed, and the differential value therebetween isoutputted as encoded data.

Then, in order to adjust the respective lengths of the encoded data to afixed length, the difference (i.e., the amount of data shortage) betweena prescribed data amount and the data amount of the encoded data at thistime point is calculated, and the shortage amount of bits among thedeleted lower bits is added (refinement is executed). In FIG. 2 ,light-gray squares represent bits that are added by the refinement.

To decode this encoded data, the bits added by the refinement are firstextracted, and a DPCM differential value in higher bits is added inorder from the right side. Thus, the higher bits in the pixel data aredecoded. The extracted bits are added to the higher bits, and further,are subjected to inverse quantization. That is, the bits lost throughencoding are replaced with prescribed values.

In other words, as a result of this encoding, information regarding bitsrepresented by dark-gray squares in FIG. 2 is lost. That is, this fixedlength encoding/decoding is executed in an irreversible way.

In such fixed length encoding and decoding, image data is encoded anddecoded in a simpler way, compared to an encoding and decoding methodsuch as AVC (Advanced Video Coding) or HEVC (High Efficiency VideoCoding). Therefore, compared to AVC, HEVC, or the like, this fixedlength encoding and decoding involve a lower load so that encoding anddecoding can be executed at higher speed. In addition, downsizing can beeasily achieved so that encoding and decoding can be executed at lowercost.

Such encoding is sometimes called simple encoding (or simplecompression). Moreover, decoding corresponding to this simple encodingis sometimes called simple decoding (or simple decompression). Simpleencoding is an image encoding technology for reducing a data transferrate and a memory band. In simple encoding, data is encoded (compressed)to keep the subjective image quality at the same level. A compressionrate of simple encoding is typically lower (for example, approximately50%) than that of general-purpose encoding such as AVC in order to keepthe subjective image quality at the same level.

In such simple encoding (simple compression) and simple decoding (simpledecompression), a code amount is a fixed length. Accordingly, comparedto a case where a code amount is variable, management of encoded data iseasy. Consequently, management of encoded data in a DRAM, into whichencoded data is recorded, for example, is also so easy that reading andwriting can be executed at higher speed and the cost can be furtherreduced.

Also, in such simple encoding (simple compression) and simple decoding(simple decompression), blocks of image data are independently encodedand decoded. Accordingly, the entire of a picture can be encoded anddecoded, and also only a part of a picture can be encoded and decoded.That is, in a case where only a part of a picture is encoded anddecoded, encoding and decoding of unnecessary data can be inhibited sothat more efficient encoding and decoding can be executed. That is, anunnecessary increase in an encoding and decoding load can be restrainedso that the processing speed can be increased and the cost can bereduced.

As described previously, information (non-encoded bit) lost through(quantization and inverse quantization in) simple encoding and simpledecoding is decompressed by an intermediate value during decoding (FIG.3 ). For example, as depicted in FIG. 3 , in a case where loss of alower 1 bit is caused by quantization, “1” is set at the lower 1 bitduring decoding. Also, in a case where loss of lower 2 bits is caused byquantization, “10 (=2)” is set at the lower 2 bits during decoding.Further, in a case where loss of lower 3 bits is caused by quantization,“100 (=4)” is set at the lower 3 bits during decoding.

When the non-encoded bits are decompressed by a prescribed value (e.g.,intermediate value) in the aforementioned manner, an input-output erroris generated. This error between an input pixel value and an outputpixel value generated through quantization is also referred to asquantization error. For example, it is assumed that a pixel value (alsoreferred to as input pixel value) of “63” (0000111111) is inputted, asdepicted on the upper side in FIG. 3 . In a case where loss of a lower 1bit is caused by quantization, “1” is set at the lower 1 bit in theaforementioned manner, and a decompressed pixel value (also referred toas output pixel value) is “63” (0000111111). That is, in this case, thequantization error is “0.”

Also, in a case where loss of lower 2 bits is caused by quantization,“10” is set at the lower 2 bits in the aforementioned manner, and thus,an output pixel value is “62” (0000111110). Therefore, the quantizationerror is “−1.” In a case where loss of lower 3 bits is caused byquantization, “100” is set at the lower 3 bits in the aforementionedmanner, and thus, an output pixel value is “60” (00001111100).Therefore, the quantization error is “−3.”

Meanwhile, it is assumed that an input pixel value is “64” (0001000000),as depicted on the lower side in FIG. 3 . In a case where loss of alower 1 bit is caused by quantization, “1” is set at the lower 1 bit inthe aforementioned manner, and thus, an output pixel value is “65”(0001000001). Therefore, the quantization error is “+1.”

Also, in a case where loss of lower 2 bits is caused by quantization,“10” is set at the lower 2 bits in the aforementioned manner, and thus,an output pixel value is “66” (0001000010). Therefore, the quantizationerror is “+2.” In a case where loss of lower 3 bits is caused byquantization, “100” is set at the lower 3 bits in the aforementionedmanner, and thus, an output pixel value is “68” (0001000100). Therefore,the quantization error is “+4.”

That is, the direction of a quantization error depends on an input pixelvalue. In contrast, in a case where a captured image is multiplied by adigital gain as described previously, a dense histogram as depicted inFIG. 1A is widened to become sparse according to the gain value, asdepicted in FIG. 1B. As a result of this widening, many pixel values areconverted to pixel values having quantization errors in the samedirection. Therefore, there is a possibility that the direction ofquantization errors is shifted toward one side. For example, in a casewhere many pixel values are distributed to multiples of 8, as depictedin FIG. 1B, the direction of quantization errors is shifted toward the +direction, as depicted in FIG. 1C.

When the direction in which quantization errors are generated is shiftedtoward one side, there is a possibility that the average value of animage (also referred to as decoded image) that is obtained bydecompression of an encoded and decoded input image (captured image), isdeviated from the average pixel value of the input image (DC deviationis generated).

When an average pixel value deviation (DC deviation) is generated, thesubjective image quality of a decoded image is deteriorated (degraded)(that is, the visual difference between a decompressed image and aninput image is increased). For example, when an average pixel value isshifted toward the + direction as in the above case, there is apossibility that the decompressed image is brighter than the inputimage.

Further, for example, in a case where an input image (captured image) isregarded as a measurement result (sensor data), there is a possibilitythat the data accuracy is deteriorated (data having lower accuracy isobtained). When the data accuracy is deteriorated, there is apossibility that an influence on subsequent processes (control,computation, etc.) using the decoded image (sensor data) is increased.For example, in a case where black-level setting is performed with acaptured image (sensor data) obtained by imaging a black image as in theexample in FIGS. 1A, 1B, and 1C, there is a possibility that a pixelvalue to be set to a black level is deviated due to the DC deviation.

It is to be noted that, when a captured image is multiplied by a digitalgain, as described previously, the pixel value difference is increasedaccording to the gain value. Consequently, the DPCM residual isincreased so that there is a possibility that the encoding efficiency isdeteriorated. Since this encoding is irreversible fixed length encodingas described previously, there is a possibility that deterioration inthe encoding efficiency leads to deterioration (degradation) in thesubjective image quality of a decoded image.

2. General Concept

<Adaptive Processing to Digital Gain>

To this end, adaptive image processing is executed on an image in whichsignal amplification has been executed, and simple encoding is executedon the image having undergone the adaptive image processing.

For example, an image processing device includes an adaptive processingsection that executes adaptive image processing of an image in whichsignal amplification has been executed, and an encoding section thatexecutes simple encoding of the image having undergone the adaptiveimage processing executed by the adaptive processing section.

As a result of this configuration, a signal value deviation (e.g., DCdeviation), which is generated through encoding and decoding of a signalgroup amplified with a digital gain, can be restrained.

More specifically, as the adaptive image processing, any one ofprocesses (any one of methods #1 to #4) described in a table in FIG. 4is executed, for example.

For example, in the method #1, in a case where an image is multiplied bya digital gain, each pixel value of the image multiplied by the digitalgain is corrected with a random offset, and then, the aforementionedsimple encoding and simple decoding are executed. As a result ofaddition of the random offset, the pixel values can be diffused. Inaddition, the value range of the offset is set according to the gainvalue of the digital gain. As a result of this, diffusion of the pixelvalues caused by the offset can be limitedly set within a prescribedrange that depends on the gain value.

Therefore, a sparse histogram in which pixel values are concentrated ata few values as in FIG. 1B, can be prevented so that the direction ofquantization errors of the respective pixel values generated throughsimple encoding and simple decoding can be inhibited from being shiftedtoward one side. That is, DC deviation can be restrained.

Consequently, when the method #1 is used, deterioration in thesubjective image quality of a decoded image can be restrained, forexample. In addition, for example, in a case where an input image(captured image) is regarded as a measurement result (sensor data),deterioration in the data accuracy can be suppressed, and an influenceon subsequent processes (control, computation, etc.) using the decodedimage (sensor data) can be restrained. For example, in a case whereblack-level setting is performed with a captured image (sensor data)obtained by imaging a black image as in the example in FIGS. 1A, 1B, and1C, the black level can be detected with higher accuracy.

Also, in the method #2, for example, in a case where an image ismultiplied by a digital gain, an offset is subtracted from the image,simple encoding and simple decoding are executed, and the offset isadded to the decoded image. As explained above with reference to FIGS.1A, 1B, 1C, 2, and 3 , the direction of a quantization error depends ona pixel value. In other words, there is a pixel value for which thequantization error is smaller than those of the other pixel values.Therefore, in a state where an offset is used to convert the pixelvalues of an image to values having small quantization errors, simpleencoding and simple decoding are executed. The pixel values of a decodedimage are restored to the respective original states. Accordingly, thequantization errors can be reduced. For example, in a case where anon-encoded bit is decompressed with an intermediate value, a pixelvalue is set to the intermediate value with use of an offset, and simpleencoding and simple decoding are executed. In this case, thequantization error ideally becomes 0.

As explained above with reference to FIGS. 1A, 1B, and 1C, when an imageis multiplied by a digital gain, the histogram is widened to includeintervals according to the gain value (a sparse state is established).Further, in many pixels, quantization errors of the pixel values aregenerated toward the same direction. That is, the direction ofquantization errors is shifted toward one side. However, when the offsetis subtracted from each pixel value in the aforementioned manner, thequantization errors become small. As a result, shift of the quantizationerrors to the one side is lessened. That is, shift of the direction ofquantization errors of pixel values generated through simple encodingand simple decoding can be restrained.

It is to be noted that a pixel value (e.g., median value) for which asmaller quantization error is generated, depends on the number of bitsto become lost through quantization. Therefore, it is sufficient thatthe value of the offset is set according to the number of bits to becomelost. That is, in this method, an offset that depends on the number ofbits to become lost through quantization is given to the image. Also, inthis method, since it is sufficient to shift a pixel value to a desiredvalue, the offset may be subtracted from a pixel value or the offset maybe added to a pixel value in the aforementioned manner.

Moreover, as a result of multiplication by a digital gain in theaforementioned manner, many pixel values are converted to values togenerate respective quantization errors toward the same direction.Therefore, an offset such as that described previously is given to (forexample, subtracted from) each pixel value so that quantization errorscan be reduced for many pixel values. That is, a shift of quantizationerrors toward one side as a whole can be inhibited. Consequently, thisoffset value is only required to be set according to the average pixelvalue (and the number of bits to become lost) of an image. As a resultof this, compared to a case an offset value is obtained for each pixel,an offset value can be easily obtained.

Also, for example, in the method #3, in a case where an image ismultiplied by a digital gain, the value range of a quantization value(qf) for use in (quantization in) simple encoding is set according tothe gain value of the digital gain. The quantization value (qf)represents a value by which a pixel value is multiplied in quantization(that is, a value representing the number of lower bits to become lost).

In general, when the quantization value (qf) is increased, the number ofbits to become lost becomes greater so that the encoding efficiency isenhanced but the subjective image quality of a decoded image isdeteriorated. Therefore, in conventional fixed length encoding such asthat disclosed in PTL 3 and 4, for example, encoding results about allthe values that the quantization value can take are verified such thatan optimum one is selected from among the values.

However, when an image is multiplied by a digital gain in theaforementioned manner, the number of lower bits of a pixel value isdegraded (an incorrect value is obtained) according to the gain value.In other words, even if these degraded lower bits become lost throughquantization, less influence of the quantization is exerted on thesubjective image quality of a decoded image (the degree of deteriorationin the image quality is substantially equal to that in a case wherequantization is not executed). Therefore, the quantization value (qf) ofbits lower than the number of bits corresponding to the gain value doesnot need to be verified (it is clearly preferable that the quantizationvalue (qf) is set to a number equal to or greater than the number ofbits corresponding to the gain value). That is, it is sufficient thatencoding results only about the quantization value (qf) of bits equal toor greater than this number of bits are verified.

That is, a limitation according to the gain value of a digital gain isimposed on the value range of a quantization value (qf). As a result ofthis, an increase in the load of verifying the aforementioned encodingresults can be restrained. That is, an increase in a load of theencoding process can be restrained.

In addition, information indicating a quantization value (qf) thusselected is contained in encoded data, and is transmitted to thedecoding side. As a result of the aforementioned limitation on the valuerange of a quantization value (qf), the quantization value (qf) can beexpressed by fewer bits (word length). That is, since the code amountcan be reduced, deterioration in the encoding efficiency can beaccordingly restrained.

Also, for example, in the method #4, in a case where an image ismultiplied by a digital gain, the digital gain is canceled (that is,division by the gain value of the digital gain is conducted), simpleencoding and simple decoding are executed, and then, the image ismultiplied again by the digital gain (multiplied by the gain value).That is, computation according to the gain value of the digital gain isexecuted. As explained above with reference to FIGS. 1A, 1B, and 1C, asa result of multiplication by a digital gain, the histogram of an imagebecomes sparse. Thus, as a result of cancel of the digital gain, simpleencoding and simple decoding can be executed while the dense state ofthe histogram is kept.

Consequently, DC deviation can be restrained. In addition, an increasein pixel value difference can be restrained so that deterioration in theencoding efficiency can be restrained.

3. First Embodiment

<Image Processing System>

Next, the methods in FIG. 4 will be more specifically explained. In thepresent embodiment, the method #1 will be explained. FIG. 5 is a blockdiagram depicting one example of a configuration according to one aspectof an image processing system to which the present technology isapplied. An image processing system 100 depicted in FIG. 5 encodes imagedata on a captured and inputted image etc. by multiplying the image databy a digital gain, records or transfers the encoded data, decodes theencoded data, and outputs image data about the decoded image.

As depicted in FIG. 5 , the image processing system 100 includes acontrol section 101, an encoding-side structure 102, and a decoding-sidestructure 103. The control section 101 executes processing related tocontrol of (the processing sections in) the encoding-side structure 102and (the processing sections in) the decoding-side structure 103. Forexample, the control section 101 sets a gain value of a digital gain(signal amplification on an image), and supplies the gain value to anamplification section 111 to amplify image data (each pixel value) withthe gain value. In addition, the control section 101 also supplies thegain value to a random offset adding section 112, an encoding section113, and a decoding section 121. It is to be noted that the controlsection 101 may be provided separately from the encoding-side structure102 and the decoding-side structure 103, as illustrated in FIG. 5 , ormay be provided in the encoding-side structure 102, or may be providedin the decoding-side structure 103.

The encoding-side structure 102 is disposed on an encoding side forencoding image data, and includes the amplification section 111, therandom offset adding section 112, and the encoding section 113, forexample.

Under control of the control section 101, the amplification section 111multiplies, by a digital gain, image data (a digital image signal)inputted to the image processing system 100. That is, the amplificationsection 111 multiplies, by a gain value supplied from the controlsection 101, each pixel value of the image data. As a result of thisprocess, for example, a histogram such as that depicted in FIG. 1A ischanged to a sparse state such as that depicted in FIG. 6A. Theamplification section 111 supplies the image data multiplied by thedigital gain to the random offset adding section 112.

Under control of the control section 101, the random offset addingsection 112 executes adaptive image processing on the image data (imagein which signal amplification has been executed) multiplied by thedigital gain. For example, the random offset adding section 112 executesthe image processing of adding, to each pixel value of the image data,an offset value which is randomly set within a value range depending onthe gain value of the digital gain. As a result of addition of therandom offset value, the histogram of the image data is changed from thesparse state such as that depicted in FIG. 6A, to a dense state such asthat depicted in FIG. 6B. The random offset adding section 112 suppliesthe image data to which the offset value has been added, to the encodingsection 113.

Under control of the control section 101, the encoding section 113executes simple encoding of the image data the histogram of which hasbeen changed to the dense state as a result of addition of the offsetvalue. For example, the encoding section 113 deletes lower bits byquantizing each block of the image data, and obtains a DPCM residual.Then, the encoding section 113 refines the lower bits, as appropriate,such that the code amounts are adjusted to a fixed length. By theencoding section 113, the fixed length encoded data thus generated isrecorded into a recording medium or is transmitted via a transmissionmedium.

The decoding-side structure 103 is disposed on a decoding side fordecoding encoded data generated by the encoding-side structure 102, andincludes a decoding section 121, for example.

Under control of the control section 101, the decoding section 121acquires the encoded data generated by the encoding section 113 via arecording medium or a transmission medium, and executes simple decodingof the encoded data. For example, the decoding section 121 extractsrefined lower bits from each block of the encoded data, and further,decompresses higher bits by inversely processing the DPCM residual sothat the lower bits lost through inverse quantization are decompressed.The decoding section 121 outputs the image data (digital image signal)thus decompressed to the outside of the image processing system 100.

<Random Offset Adding Section>

FIG. 7 is a block diagram depicting a main configuration example of therandom offset adding section 112. As depicted in FIG. 7 , the randomoffset adding section 112 includes a pseudo random number generationsection 141, a value range limiting section 142, a computing section143, and a clipping section 144.

The pseudo random number generation section 141 executes a processregarding generation of a pseudo random number. For example, the pseudorandom number generation section 141 receives a prescribed initial valueas an input, and generates a pseudo random number for each inputtedpixel value. For example, the pseudo random number generation section141 generates an 11-bit pseudo random number, and supplies the pseudorandom number to the value range limiting section 142. The number ofbits of the pseudo random number is arbitrarily defined.

The value range limiting section 142 executes a process regardingimposing a limitation on the value range of a pseudo random number. Forexample, the value range limiting section 142 receives, as inputs, an11-bit pseudo random number supplied from the pseudo random numbergeneration section 141 and the gain value (gain) of a digital gainsupplied from the control section 101, and limits the value range of thepseudo random number to a range according to the gain value (correctsthe pseudo random number to fall within the range). For example, thevalue range limiting section 142 corrects the value range of the pseudorandom number on the basis of a syntax such as that depicted in FIG. 8 .

In the process is executed in accordance with the syntax in FIG. 8 , thevalue range limiting section 142 limits the value range of the pseudorandom number to a range from “−gain/2” to “gain/2.” That is, forexample, in a case where the gain value is an odd number (for example,gain=7), the value range limiting section 142 sets a random offset valuewithin the range from “−gain/2” to “gain/2,” as in a histogram depictedin FIG. 9A. In this case, the number of offset values at each value from“−gain/2” to “gain/2” is identical (fixed).

Also, for example, in a case where the gain value is an even number (forexample, gain=8), the value range limiting section 142 sets a randomoffset value within a range from “−gain/2” to “gain/2,” as in ahistogram depicted in FIG. 9B. That is, in this case, the number ofoffset values at each value from “−gain/2+1” to “gain/2−1” is identical(fixed). In the case where the gain value is an even number, “−gain/2”and “gain/2” each overlap the value range of the corresponding adjacentoffset value. Therefore, in this case, the number of offset values ateach of “−gain/2” and “gain/2” is half of that at the remaining values,as depicted in FIG. 9B.

The value range limiting section 142 supplies, as an offset value, thepseudo random number the value range of which has been thus limited, tothe computing section 143.

The computing section 143 gives (for example, adds), to each input pixelvalue, the offset value supplied from the value range limiting section142. For example, in a case where the gain value is an odd number (gain% 2 ≠0), the median pixel value in the value range of the offset valueis equally dispersed in the range from “−gain/2” to “gain/2,” as aresult of addition of the offset value. Also, for example, in a casewhere the gain value is an even number (gain % 2=0), the median pixelvalue in the value range of the offset value is dispersed in the rangefrom “−gain/2” to “gain/2” in such a manner depicted in FIG. 9B, as aresult of addition of the offset value. Accordingly, the dense histogramis obtained, as depicted in FIG. 6B. That is, the computing section 143adds, as an offset value, the pseudo random number corrected to fallwithin the value range according to the gain value, to each pixel value(input pixel value) of the image. The computing section 143 supplies theimage data to which the offset value has been given, to the clippingsection 144.

The clipping section 144 executes clipping to adjust a pixel value thatis outside a prescribed range (e.g., a range from 0 to 1023), to theupper limit value (1023) or the lower limit value (0) such that, forexample, the bit length is adjusted to 10 bits. The clipping section 144supplies the image data (output pixel value) having undergone theclipping, to the encoding section 113.

As described previously, simple encoding is executed after a randomoffset value is added to image data so that simple encoding and simpledecoding can be executed while the histogram is in a dense state.Consequently, shift of quantization errors of respective pixel values toone side through simple encoding and simple decoding can be inhibited.That is, when the method #1 is adopted, the image processing system 100can restrain an average signal value deviation (DC deviation) which isgenerated through encoding and decoding of an amplified single group.

It is to be noted that an influence which is exerted on the subjectiveimage quality of a decoded image is small because, even if an offsetvalue is added in the aforementioned manner, only lower bits thatinclude errors mainly due to a digital gain are changed. That is, whilethe influence which is exerted on the subjective image quality of adecoded image is restrained, DC deviation which is generated throughencoding and decoding can be restrained.

<Flow of Encoding Process>

Next, the flow of a process which is executed in the image processingsystem 100 will be explained. First, an example of the flow of anencoding process which is executed in the encoding-side structure 102will be explained with reference to a flowchart in FIG. 10 .

When the encoding process is started, the amplification section 111 onthe encoding side of the image processing system 100 multiplies, at stepS101, inputted image data by a digital gain having a gain value set bythe control section 101.

At step S102, the random offset adding section 112 adds a random offsetto the image data according to the gain value of the digital gain atstep S101.

At step S103, the encoding section 113 executes simple encoding of theimage data to which the random offset has been added. For example, theencoding section 113 executes simple encoding of each block of the imagedata.

At step S104, the encoding section 113 outputs, in the form of a bitstream, for example, the encoded data generated by simple encoding. Forexample, by the encoding section 113, the bit stream is recorded into arecording medium or is transmitted via a transmission medium.

When step S104 is completed, the encoding process is ended.

<Flow of Offset Addition Process>

Next, an example of the flow of an offset addition process of adding arandom offset to a pixel value will be explained with reference to aflowchart in FIG. 11 .

When the offset addition process is started, the pseudo random numbergeneration section 141 generates, at step S121, a pseudo random numberas an offset to be given to an input pixel value.

At step S122, the value range limiting section 142 imposes a limitationon the value range of the offset (pseudo random number) set at stepS121, according to the gain value of the digital gain.

At step S123, the computing section 143 adds, to each pixel value of theimage, the offset (pseudo random number) the value range of which hasbeen set at step S122. In addition, the clipping section 144 executesclipping on the addition result, thereby converting the addition resultto data of a prescribed bit length (e.g., 10 bits).

When step S123 is completed, the offset addition process is ended. Then,the process returns to FIG. 10 .

<Flow of Decoding Process>

Next, an example of the flow of a decoding process which is executed inthe decoding-side structure 103 will be explained with reference to aflowchart in FIG. 12 .

When the decoding process is started, the decoding section 121 in thedecoding-side structure 103 acquires, at step S141, the bitstream(encoded data) generated by the encoding-side structure 102 via arecording medium or a transfer medium.

At step S142, the decoding section 121 executes simple decoding of thebitstream acquired at step S141. For example, the decoding section 121executes simple decoding of each block of the bitstream.

When step S142 is completed, the decoding process is ended.

By executing the aforementioned processes, the image processing system100 can encode and decode an amplified signal group by the method #1.Accordingly, the image processing system 100 can restrain an averagesignal value deviation (DC deviation) which is generated throughencoding and decoding of an amplified signal group.

Consequently, the image processing system 100 can deterioration in theaccuracy of data (as a measurement result), for example. In addition,for example, deterioration in the subjective image quality of a decodedimage can be restrained.

<Another Configuration of Image Processing System>

It is to be noted that the configuration of the image processing system100 is not limited to the example in FIG. 5 . For example, as depictedin FIG. 13 , encoded data (a bitstream) may be transmitted from theencoding-side structure 102 to the decoding-side structure 103 throughcommunication based on a prescribed communication scheme.

In this case, the image processing system 100 further includes atransmission section 171 in the encoding-side structure 102, as depictedin FIG. 13 . In addition, the image processing system 100 furtherincludes a reception section 172 in the decoding-side structure 103.

In the encoding-side structure 102, the encoding section 113 suppliesgenerated encoded data (a bitstream) to the transmission section 171.

The transmission section 171 and the reception section 172 areprescribed communication interfaces that exchange information byperforming communication by a scheme conforming to a prescribedcommunication standard. For example, the transmission section 171converts (for example, packetizes) a bitstream supplied from theencoding section 113, to transmission data of a format conforming to thecommunication standard, and supplies the transmission data to thereception section 172 via a prescribed transmission path. The receptionsection 172 receives the transmission data (e.g., packets) of theprescribed format, and decompresses the encoded data. The receptionsection 172 supplies the decompressed encoded data to the decodingsection 121.

In the aforementioned manner, encoded data (a bitstream) havingundergone simple encoding can be transmitted from an encoding side to adecoding side by a scheme conforming to a prescribed communicationstandard. Consequently, for example, an existing communication standardcan be adopted as the communication standard in this case, anddevelopment thereof can be facilitated.

4. Second Embodiment

<Image Processing System>

In the present embodiment, the method #2 in FIG. 4 will be explained.FIG. 14 is a block diagram depicting one example of a configurationaccording to one aspect of an image processing system to which thepresent technology is applied.

In FIG. 14 , the encoding-side structure 102 of the image processingsystem 100 includes the amplification section 111, a subtraction offsetsetting section 211, a computing section 212, a clipping section 213,and the encoding section 113.

The subtraction offset setting section 211 executes a process regardingsetting of a subtraction offset. A subtraction offset is to besubtracted from each pixel value of image data multiplied by a digitalgain by the amplification section 111. The subtraction offset settingsection 211 sets such a subtraction offset on the basis of image datamultiplied by a digital gain by the amplification section 111. Morespecifically, the subtraction offset setting section 211 sets asubtraction offset value on the basis of the average pixel value ofimage data multiplied by a digital gain and a quantization value (aquantization value of quantization which is executed in simple encoding)of simple encoding which is executed by the encoding section 113. Thesubtraction offset setting section 211 supplies the set subtractionoffset to the computing section 212.

The computing section 212 executes adaptive image processing ofsubtracting the subtraction offset set by the subtraction offset settingsection 211, from each pixel value of the image data multiplied by thedigital gain by the amplification section 111. The computing section 212supplies the subtraction result to the clipping section 213.

The clipping section 213 executes clipping of the supplied subtractionresult (image data which has been multiplied by the digital gain andfrom which the subtraction offset has been subtracted), and clips thelower limit thereof (e.g., 0). The clipping section 213 supplies theclipped image data to the encoding section 113.

The encoding section 113 executes simple encoding of the image datasupplied from the clipping section 213. By the encoding section 113, thefixed length encoded data thus generated is recorded into a recordingmedium or is transmitted via a transmission medium.

Moreover, in FIG. 14 , the decoding-side structure 103 includes thedecoding section 121, an addition offset setting section 221, acomputing section 222, and a clipping section 223.

The addition offset setting section 221 executes a process regardingsetting of an addition offset. An addition offset is to be added to eachpixel value of image data decompressed by the decoding section 121. Theaddition offset setting section 221 sets an addition offset by a methodbasically similar to that of the subtraction offset setting section 211.For example, the addition offset setting section 221 sets an additionoffset on the basis of the decompressed image data. More specifically,the addition offset setting section 221 sets an addition offset value onthe basis of the average pixel value of the decompressed image data anda quantization value (the quantization value supplied from the encodingside) of quantization (simple encoding) which is executed by theencoding section 113. The addition offset setting section 221 suppliesthe set addition offset to the computing section 222.

The computing section 222 executes adaptive processing of adding theaddition offset supplied from the addition offset setting section 221,to each pixel value of the decompressed image data supplied from thedecoding section 121. The computing section 222 supplies the additionresult to the clipping section 223.

The clipping section 223 executes clipping of the supplied subtractionresult (the image data which has been decompressed and to which theaddition offset has been added), and clips the upper limit (maximumvalue) thereof. The clipping section 223 outputs the clipped image datato the outside of the image processing system 100.

In this case, as described previously with reference to FIG. 4 , as aresult of subtraction of the subtraction offset from each pixel value ofthe image data multiplied by the digital gain in the encoding-sidestructure 102, the pixel values are shifted to values for which smallerquantization errors are generated. In other words, the subtractionoffset is set to a value to achieve this shift.

Then, as a result of addition of the addition offset to each pixel valueof the decompressed image data in the decoding-side structure 103, thepixel values are shifted to the original values (that is, the shift ofthe pixel values using the subtraction offset in the encoding-sidestructure 102 is canceled). In other words, the addition offset is setto a value to achieve this cancel.

Through the aforementioned processes, simple encoding and simpledecoding can be executed while quantization errors are made smaller.Therefore, as a result, a shift of the direction of quantization errorsto one side can be lessened. That is, a shift of the direction ofquantization errors of pixel values to one side due to encoding anddecoding can be restrained.

<Subtraction Offset Setting Section>

FIG. 15 is a block diagram depicting a main configuration example of thesubtraction offset setting section 211. As depicted in FIG. 15 , thesubtraction offset setting section 211 includes an average valuemeasuring section 231, an offset value selection section 232, and anoffset value supply section 233.

The average value measuring section 231 calculates the average pixelvalue of a frame (t−1) preceding a process target frame (current framet) of image data supplied from the amplification section 111. Theaverage value measuring section 231 supplies the calculated averagepixel value to the offset value selection section 232.

The offset value selection section 232 sets an offset value (asubtraction offset) on the basis of the average pixel value of the frame(t−1) supplied from the average value measuring section 231 and amaximum quantization bit loss amount that is determined in accordancewith a compression rate of simple encoding.

As described previously, an offset value to make quantization errorssmaller depends on the average pixel value of image data multiplied by adigital gain and a maximum bit loss amount in quantization. For example,in a case where image data corresponds to the histogram depicted in FIG.1B, the value of a subtraction offset to make quantization errorssmaller can be obtained on the average pixel value of the image and themaximum bit loss amount, as illustrated in a table in FIG. 16 .

That is, the offset value selection section 232 holds the table inadvance, and obtains the value of a subtraction offset with reference tothe table. Accordingly, the offset value selection section 232 can moreeasily set a subtraction offset. The offset value selection section 232supplies the set subtraction offset to the offset value supply section233.

The offset value supply section 233 supplies, as a subtraction offsetfor the current frame (t), the subtraction offset supplied from theoffset value selection section 232, to the computing section 212.

It is to be noted that an average pixel value may be calculated by usinga frame that is previous to the current frame by two or more. That is, asubtraction offset may be calculated by using a frame that is previousto the current frame by two or more. However, when a frame that iscloser to the current frame is used to obtain the average pixel value, asubtraction offset of a more accurate value (a value to makequantization errors smaller) is likely to be obtained.

In addition, a subtraction offset may be set for each of colors (forexample, for each of R, G, and B) in image data. In this case, theaverage value measuring section 231 may calculate an average pixel valuefor each color, and the offset value selection section 232 may set asubtraction offset value for each color by using the average pixel valuecalculated for each color. As a result of this, a subtraction offset ofa more accurate value (a value to make quantization errors smaller) canbe easily obtained. For example, even in a case where the maximum bitloss amounts for respective colors are different from one another, anoffset value to make quantization errors smaller can be obtained in theaforementioned manner.

It is to be noted that the addition offset setting section 221 also hasa configuration similar to that of the subtraction offset settingsection 211, and sets an addition offset by a method similar to that forthe subtraction offset setting section 211. Thus, an explanation thereofis omitted.

As described so far, when the method #2 is adopted, the image processingsystem 100 can restrain an average signal value deviation (DC deviation)which is generated through encoding and decoding of an amplified signalgroup.

<Flow of Encoding Process>

Next, the flow of a process which is executed in the image processingsystem 100 will be explained. First, an example of the flow of anencoding process which is executed in the encoding-side structure 102will be explained with reference to the flowchart in FIG. 17 .

When the encoding process is started, the amplification section 111which is an encoding-side section of the image processing system 100multiplies, at step S201, inputted image data by a digital gain of again value set by the control section 101.

At step S202, the subtraction offset setting section 211 obtains andsets a subtraction offset in the aforementioned manner.

At step S203, the computing section 212 subtracts the subtraction offsetfrom the image data multiplied by the digital gain at step S201.

At step S204, the clipping section 213 executes clipping of thesubtraction result calculated at step S203, that is, the image datawhich has been multiplied by the digital gain and from which thesubtraction offset has been subtracted, thereby clips the lower limit ofeach pixel value.

At step S205, the encoding section 113 executes simple encoding of theimage data the lower limit of which has been clipped.

At step S206, the encoding section 113 outputs, in the form of abitstream, for example, the encoded data generated through simpleencoding. By the encoding section 113, the bitstream is recorded into arecording medium or is transmitted via a transmission medium, forexample.

When step S206 is completed, the encoding process is ended.

<Flow of Offset Value Setting Process>

Next, the flow of an offset value setting process, which executed atstep S202 in FIG. 17 , of setting a subtraction offset will be explainedwith reference to a flowchart in FIG. 18 .

When the offset value setting process is started, the offset valuesupply section 233 supplies and sets a subtraction offset for thecurrent frame to the computing section 212 at step S221. The subtractionoffset is an offset value (offset value that has been already set) setby the past process (for example, when the frame preceding the currentframe was a process target). That is, the offset value is set on thebasis of image data on the frame (t−1) preceding the current frame (t).

At step S222, the average value measuring section 231 calculates theaverage pixel value of the image data on the current frame.

At step S223, the offset value selection section 232 selects (sets) asubtraction offset for a frame succeeding to the current frame, withreference to the table in FIG. 16 , for example, on the basis of theaverage pixel value calculated at step S222 and the maximum bit lossamount of the current frame calculated according to the compressionrate. The offset value supply section 233 holds the subtraction offsetuntil the next frame is processed. Then, at step S221 for the nextframe, the offset value supply section 233 supplies the subtractionoffset to the computing section 212.

When step S223 is completed, the offset value setting process is ended.Then, the process returns to FIG. 17 .

<Flow of Decoding Process>

Next, an example of the flow of a decoding process which is executed inthe decoding-side structure 103 will be explained with reference to aflowchart in FIG. 19 .

When the decoding process is started, the addition offset settingsection 221 in the decoding-side structure 103 obtains and sets anaddition offset at step S241. It is to be noted that the addition offsetis set in accordance with a flow similar to that for setting asubtraction offset (the offset value setting process), which has beenexplained previously with reference to the flowchart in FIG. 18 . Thus,an explanation thereof will be omitted.

At step S242, the decoding section 121 acquires a bitstream (encodeddata) generated in the encoding-side structure 102, via a recordingmedium or a transmission medium.

At step S243, the decoding section 121 executes simple decoding of thebitstream acquired at step S242.

At step S244, the computing section 222 adds the addition offset set atstep S241 to the decoded image generated as a result of step S243.

At step S245, the clipping section 223 clips the upper limit of thedecoded image to which the addition offset has been added at step S244.

When step S245 is completed, the decoding process is ended.

By executing the aforementioned processes, the image processing system100 can execute simple encoding and simple decoding of an amplifiedsignal group by the method #2. Accordingly, the image processing system100 can restrain an average signal value deviation (DC deviation) whichis generated through encoding and decoding of an amplified signal group.

Consequently, the image processing system 100 can restrain deteriorationin the accuracy of data (as a measurement result). In addition, forexample, deterioration in the subjective image quality of a decodedimage can be restrained.

<Another Configuration Example of Subtraction Offset Setting Section>

It is to be noted that a bit loss amount may be calculated from imagedata, and a subtraction offset may be set by use of the calculated bitloss amount.

FIG. 20 is a block diagram depicting another example of the subtractionoffset setting section 211. In this case, the subtraction offset settingsection 211 includes a compression section 251 and an average valuemeasuring section 252, in addition to the average value measuringsection 231 to the offset value supply section 233, as depicted in FIG.20 .

The compression section 251 compresses (that is, executes simpleencoding of) a frame (t−1) preceding a process target frame (currentframe (t)) of image data supplied from the amplification section 111, ina manner similar to that of the encoding section 113, and obtains aquantization value (bit loss amount) of each pixel value. Thecompression section 251 supplies the calculated quantization value (bitloss amount) of each pixel value to the average value measuring section252. It is to be noted that the bit loss amount may be calculated byusing a frame that is previous to the current frame by two or more. Thatis, the subtraction offset may be calculated by using a frame that isprevious to the current frame by two or more.

The average value measuring section 252 calculates the average value(average quantization value (bit loss amount)) of supplied quantizationvalues of respective pixels. That is, the average value measuringsection 252 calculates, for a frame (e.g., the frame (t−1) preceding thecurrent frame (t)) preceding a process target frame (current frame (t))of the image data supplied from the amplification section 111, theaverage value of quantization values of pixels in simple image encoding.The average value measuring section 252 supplies the calculated averagequantization value to the offset value selection section 232.

The offset value selection section 232 sets a subtraction offset on thebasis of the average pixel value supplied from the average valuemeasuring section 231 and the average quantization value supplied fromthe average value measuring section 252. A method for setting thesubtraction offset is basically similar to that in the case of FIG. 15 ,and is based on the table in FIG. 16 , for example.

<Flow of Offset Value Setting Process>

An example of the flow of an offset value setting process in this casewill be explained with reference to a flowchart in FIG. 21 .

When the offset value setting process is started, the offset valuesupply section 233 supplies and sets, at step S261, a subtraction offsetfor the current frame to the computing section 212. The subtractionoffset is an offset value (offset value that has been already set) setby the past process (for example, when the frame preceding the currentframe was a process target). That is, the offset value is set on thebasis of the image data on the frame (t−1) preceding the current frame(t).

At step S262, the average value measuring section 231 calculates anaverage pixel value of the image data on the current frame.

At step S263, the compression section 251 compresses the image data onthe current frame in the manner similar to that of simple encoding whichis executed by the encoding section 113, and obtains quantization valuesof respective pixels.

At step S264, the average value measuring section 252 obtains theaverage value (average quantization value) of the quantization valuescalculated at step S263.

At step S265, the offset value selection section 232 selects (sets) asubtraction offset for a frame next to the current frame with referenceto the table in FIG. 16 , for example, on the basis of the average pixelvalue calculated at step S262 and the average quantization valuecalculated at step S264. The offset value supply section 233 holds thesubtraction offset until the next frame is processed. Then, at step S261for the next frame, the offset value supply section 233 supplies thesubtraction offset to the computing section 212.

When step S265 is completed, the offset value setting process is ended.Then, the process proceeds to FIG. 17 .

As described so far, also in this case, the image processing system 100can restrain an average signal value deviation (DC deviation) which isgenerated through encoding and decoding of an amplified signal group.

It is to be noted that, in the above explanation, image data on thecurrent frame is processed and a subtraction offset for the next frameis set, but the subtraction offset setting section 211 may be configuredto hold image data on one frame, and set a subtraction offset for thecurrent frame by using image data on a frame preceding the currentframe. The similar setting applies to an addition offset.

<Another Configuration of Image Processing System>

It is to be noted that the configuration of the image processing system100 is not limited to the example in FIG. 14 . For example, encoded data(a bitstream) may be transmitted from the encoding-side structure 102 tothe decoding-side structure 103 through communication based on aprescribed communication scheme, as depicted in FIG. 22 .

In this case, the image processing system 100 further includes thetransmission section 171 in the encoding-side structure 102, as depictedin FIG. 22 . In addition, the image processing system 100 furtherincludes the reception section 172 in the decoding-side structure 103.

That is, for example, the transmission section 171 converts (forexample, packetizes) a bitstream supplied from the encoding section 113,to transmission data of a format conforming to the communicationstandard, and supplies the transmission data to the reception section172 via a prescribed transmission path. The reception section 172receives the transmission data (e.g., packets) of the prescribed format,and decompresses the encoded data. The reception section 172 suppliesthe decompressed encoded data to the decoding section 121.

As a result of this, encoded data (a bitstream) having undergone simpleencoding can be transmitted from an encoding side to a decoding side bya scheme conforming to a prescribed communication standard.Consequently, for example, an existing communication standard can beadopted as the communication standard in this case, and developmentthereof can be facilitated.

5. Third Embodiment

<Image Processing System>

In the present embodiment, the method #3 in FIG. 4 will be explained.FIG. 23 is a block diagram depicting one example of a configurationaccording to one aspect of an image processing system to which thepresent technology is applied.

In FIG. 23 , the encoding-side structure 102 of the image processingsystem 100 includes the amplification section 111, a quantization valuerange setting section 311, and the encoding section 113.

The quantization value range setting section 311 executes a processregarding setting of a quantization value range. For example, thequantization value range setting section 311 sets the range of aquantization value (qf) of (quantization which is executed in) simpleencoding, according to the gain value of a digital gain supplied fromthe control section 101. As described previously with reference to FIG.4 , a limitation that depends on the gain value of the digital gain canbe imposed on the value range of a quantization value (qf).

For example, the quantization value range setting section 311 previouslyhas information regarding a table such as that depicted in FIG. 24 , andsets the value range of a quantization value (qf) that corresponds tothe gain value (i.e., the gain value of single amplification executed onan image) supplied from the control section 101, with reference to thetable.

For example, in a case where image data is multiplied by a digital gainof an eight-fold gain value, as in FIG. 25A, information regarding lower3 bits are degraded due to the digital gain, as depicted in FIG. 25B.Accordingly, while deterioration in the subjective image quality of adecoded image is restrained, loss of these lower 3 bits can be achievedthrough quantization. That is, a limitation is imposed such that thevalue range of a quantization value (qf) is changed from 0-9 to 3-9(even if such a limitation is imposed, deterioration in the subjectiveimage quality of a decoded image can be restrained).

Such a limitation is imposed on the value range of a quantization value(qf), whereby verification of encoding results can be omitted for thepart where the limitation has been imposed on the value range of thequantization value (qf). Accordingly, an increase in the simple encodingload can be restrained.

In addition, FIG. 26 is a diagram depicting a main configuration exampleof encoded data. Encoded data 341, which is depicted in FIG. 26 ,contains information (the value of qf) representing a quantization value(qf) (a hatched part in FIG. 26). As described previously, when alimitation is imposed on the value range of a quantization value (qf),the quantization value (qf) can be expressed with fewer bits (wordlength). Accordingly, the code amount of information representing thequantization value (qf) in the encoded data can be suppressed. That is,deterioration in the encoding efficiency can be restrained, anddeterioration in the subjective image quality of a decoded image can berestrained.

After setting the value range of a quantization value (qf), thequantization value range setting section 311 supplies informationregarding the value range, the image data, etc. to the encoding section113.

The encoding section 113 executes simple encoding of the image data inaccordance with the supplied value range of a quantization value (qf).That is, the encoding section 113 verifies encoding results for thelimited value range of a quantization value (qf), and selects an optimumone. In addition, through simple encoding, the encoding section 113generates encoded data containing information regarding the value rangeof a quantization value (qf).

By the encoding section 113, the generated fixed length encoded data isrecorded into a recording medium or is transmitted via a transmissionmedium.

The decoding-side structure 103 is disposed on a decoding-side fordecoding encoded data generated by the encoding-side structure 102, andincludes the decoding section 121, for example.

Under control of the control section 101, the decoding section 121acquires the encoded data generated by the encoding section 113, via arecording medium or a transmission medium, and executes simple decodingof the encoded data.

During the simple decoding, the decoding section 121 makes reference toinformation regarding the value range of a quantization values (qf)contained in the encoded data, and executes inverse quantization on thebasis of the information (on the basis of the value ranges of aquantization value (qf) indicated by the information). The decodingsection 121 outputs the image data (digital image signal) thusdecompressed, to the outside of the image processing system 100.

As described so far, when the method #3 is adopted, the image processingsystem 100 can restrain an increase in the encoding process load, andfurther, can restrain deterioration in the encoding efficiency.

<Flow of Encoding Process>

An example of the flow of an encoding process which is executed in theencoding-side structure 102 in this case will be explained with aflowchart in FIG. 27 .

When the encoding process is started, the amplification section 111which is an encoding-side section of the image processing system 100multiplies, at step S301, inputted image data by a digital gain of again value set by the control section 101.

At step S302, the quantization value range setting section 311 sets therange (value range) of a quantization value (qf) of simple encoding,according to the gain value of the digital gain.

At step S303, the encoding section 113 executes simple encoding of theimage data multiplied by the digital gain at step S301, in accordancewith the range (value range) of a quantization value (qf) set at stepS302.

At step S304, the encoding section 113 outputs, in the form of abitstream, for example, the encoded data thus generated. By the encodingsection 113, the bitstream is recorded into a recording medium or istransmitted via a transmission medium, for example.

When step S304 is completed, the encoding process is ended.

<Flow of Decoding Process>

Next, an example of the flow of a decoding process which is executed inthe decoding-side structure 103 will be explained with reference to aflowchart in FIG. 28 .

When the decoding process is started, the decoding section 121 in thedecoding-side structure 103 acquires, at step S321, a bitstream (encodeddata) generated in the encoding-side structure 102, via a recordingmedium or a transmission medium.

At step S322, the decoding section 121 executes simple decoding of thebitstream acquired at step S321. For example, the decoding section 121executes simple decoding of each block of the bitstream (encoded data).

Here, the decoding section 121 makes reference to information regardingthe value range of a quantization value (qf) contained in the encodeddata, and executes inverse quantization on the basis of the information(on the basis of the value range of a quantization value (qf) indicatedby the information). The decoding section 121 outputs the image data(digital image signal) thus decompressed, to the outside of the imageprocessing system 100.

When step S322 is completed, the decoding process is ended.

By executing the aforementioned processes, the image processing system100 can execute simple encoding and simple decoding of an amplifiedsignal group by the method #3. Consequently, the image processing system100 can restrain an increase in the encoding process load, and further,can restrain deterioration in the encoding efficiency.

<Another Configuration of Image Processing System>

It is to be noted that the configuration of the image processing system100 is not limited to the example in FIG. 23 . For example, encoded data(a bitstream) may be transmitted from the encoding-side structure 102 tothe decoding-side structure 103 through communication based on aprescribed communication scheme, as depicted in FIG. 29 .

In this case, the image processing system 100 further includes thetransmission section 171 in the encoding-side structure 102, as depictedin FIG. 29 . In addition, the image processing system 100 furtherincludes the reception section 172 in the decoding-side structure 103.

That is, for example, the transmission section 171 converts (forexample, packetizes) the bitstream supplied from the encoding section113, to transmission data of a format conforming to the communicationstandard, and supplies the transmission data to the reception section172 via a prescribed transmission path. The reception section 172receives the transmission data (e.g., packets) of the prescribed format,and decompresses the encoded data. The reception section 172 suppliesthe decompressed data to the decoding section 121.

In the manner described so far, encoded data (a bitstream) generatedthrough simple encoding can be transmitted from the encoding side to thedecoding side by a scheme conforming to a prescribed communicationstandard. Consequently, for example, an existing communication standardcan be adopted as the communication standard in this case, anddevelopment thereof can be facilitated.

6. Fourth Embodiment

<Image Processing System>

In the present embodiment, the method #4 in FIG. 4 will be explained.FIG. 30 is a block diagram depicting one example of a configurationaccording to one aspect of an image processing system to which thepresent technology is applied. In FIG. 30 , the encoding-side structure102 of the image processing system 100 includes the amplificationsection 111, a computing section 411, and the encoding section 113.

The computing section 411 divides the image data supplied from theamplification section 111, that is, the image data multiplied by thedigital gain, by the gain value of the digital gain supplied from thecontrol section 101. That is, the computing section 411 cancels thedigital gain by which the image data has been multiplied. Accordingly,the encoding section 113 executes simple encoding of the image data inwhich the digital gain has been canceled.

That is, simple encoding of the image data is executed while a densestate such as that depicted in FIG. 1A is established, for example.Accordingly, DC deviation which is generated through the simple encodingcan be restrained. In addition, an increase in the pixel valuedifference generated by multiplication by a digital gain is alsorestrained. Thus, an increase in the DPCM residual can be restrained andan increase in the encoding efficiency can be restrained.

By the encoding section 113, the generated fixed length encoded data isrecorded into a recording medium or is transmitted via a transmissionmedium.

In addition, in FIG. 30 , the decoding-side structure 103 includes thedecoding section 121 and a computing section 421. The computing section421 multiplies the image data (each pixel value of the image data)decompressed by the decoding section, by a gain value of a digital gainsupplied from the control section 101. That is, image data which hasbeen multiplied by the digital gain is obtained. The computing section421 outputs the image data which has been multiplied by the digitalgain, to the outside of the image processing system 100.

As a result of this, the image processing system 100 can execute simpleencoding and simple decoding of image data which has not been multipliedby a digital gain. That is, the influence of simple encoding and simpledecoding can be inhibited from being exerted on image data which hasbeen multiplied by a digital gain. Consequently, DC deviation can berestrained, and an increase in the encoding efficiency can berestrained.

<Flow of Encoding Process>

Next, an example of an encoding process which is executed in theencoding-side structure 102 in this case will be explained withreference to a flowchart in FIG. 31 .

When the encoding process is started, the amplification section 111 inthe encoding-side structure 102 of the image processing system 100multiplies, at step S401, inputted image data by a digital gain of again value set by the control section 101.

At step S402, the computing section 411 divides, by a gain value set bythe control section 101, the image data multiplied by the digital gain,thereby cancels the digital gain.

At step S403, the encoding section 113 executes simple encoding of theimage data in which the digital gain has been canceled. For example, theencoding section 113 executes simple encoding of each block of the imagedata.

At step S404, the encoding section 113 outputs, in the form of abitstream, for example, the encoded data generated through the simpleencoding. By the encoding section 113, the bitstream is recorded into arecording medium or is transmitted via a transmission medium, forexample.

When step S404 is completed, the encoding process is ended.

<Flow of Decoding Process>

Next, an example of the flow of a decoding process which is executed inthe decoding-side structure 103 will be explained with reference to aflowchart in FIG. 32 .

When the decoding process is started, the decoding section 121 in thedecoding-side structure 103 acquires, at step S421, a bitstream (encodeddata) generated by the encoding-side structure 102, via a recordingmedium or a transmission medium.

At step S422, the decoding section 121 executes simple decoding of thebitstream acquired at step S421. For example, the decoding section 121executes simple decoding of each block of the encoded data.

At step S423, the computing section 421 multiplies the decoded imagegenerated through simple decoding, by a digital gain of a gain value setby the control section 101.

When step S423 is completed, the decoding process is ended.

By executing the processes in the aforementioned manner, the imageprocessing system 100 can encode and decode an amplified signal group bythe method #4. Accordingly, the image processing system 100 can restrainan average signal value deviation (DC deviation).

Consequently, the image processing system 100 can restrain deteriorationin the accuracy of data (as a measurement result), for example. Inaddition, deterioration in the subjective image quality of a decodedimage can be restrained, for example.

In addition, the image processing system 100 can restrain an increase inthe pixel value difference so that deterioration in the encodingefficiency can be restrained. It is to be noted that, although simpleencoding of conducting division by a gain value after signalamplification has been explained above, the simple encoding is notlimited to this. For example, simple encoding in which division by again value is conducted but signal amplification (that is,multiplication by a digital gain) is omitted may be executed. As aresult of this, an increase in the encoding process load can berestrained.

<Another Configuration of Image Processing System>

It is to be noted that the configuration of the image processing system100 is not limited to the example in FIG. 30 . For example, encoded data(a bitstream) may be transmitted from the encoding-side structure 102 tothe decoding-side structure 103 through communication based on aprescribed communication scheme, as depicted in FIG. 33 .

In this case, the image processing system 100 further includes atransmission section 171 in the encoding-side structure 102, as depictedin FIG. 33 . In addition, the image processing system 100 furtherincludes a reception section 172 in the decoding-side structure 103.

That is, for example, the transmission section 171 converts (forexample, packetizes) a bitstream supplied from the encoding section 113,to transmission data of a format conforming to the communicationstandard, and supplies the transmission data (e.g., packets) to thereception section 172 via a prescribed transmission path. The receptionsection 172 receives the transmission data (e.g., packets) of theprescribed format, and decompresses the encoded data. The receptionsection 172 supplies the encoded data thus decompressed, to the decodingsection 121.

In the manner described so far, encoded data (a bitstream) havingundergone simple encoding can be transmitted from an encoding side to adecoding side by a scheme conforming to a prescribed communicationstandard. Consequently, for example, an existing communication standardcan be adopted as the communication standard in this case, anddevelopment thereof can be facilitated.

7. Fifth Embodiment

<Application Example: Imaging Element>

Next, an example of applying the present technology described so far toa certain device will be explained. FIG. 34 is a block diagram depictinga main configuration example of a stacked image sensor 510 to which thepresent technology is applied. The stacked image sensor 510 depicted inFIG. 34 is an image sensor (imaging element) that captures an image of asubject, obtains digital data (image data) about the captured image, andoutputs the image data.

As depicted in FIG. 34 , the stacked image sensor 510 includes threesemiconductor substrates 511 to 513. These semiconductor substrates thatare in a state of being stacked are sealed to be formed into a module(formed integrally). That is, these semiconductor substrates constitutea multiplayer structure (stacked structure). Electronic circuits areformed on the respective semiconductor substrates 511 to 513. Thecircuits formed on the respective semiconductor substrates are connectedto each other through vias etc. A path between the (circuits formed onthe) semiconductor substrates is also referred to as bus. For example,through a bus 521, data, etc. can be exchanged between the circuit onthe semiconductor substrate 511 and the circuit on the semiconductorsubstrate 512. Also, through a bus 522, data, etc. can be exchangedbetween the circuit on the semiconductor substrate 512 and the circuiton the semiconductor substrate 513.

Further, an interface 523 of the stacked image sensor 510 is formed onthe circuit formed on the semiconductor substrate 512. That is, throughthe interface 523, the circuit formed on the semiconductor substrate 512can exchange data etc. with a circuit (e.g., a circuit formed on acircuit substrate 530) external to the stacked image sensor 510.Communication based on a communication scheme conforming to a prescribedcommunication standard is performed through the interface 523. Thecommunication standard is arbitrarily defined. For example, MIPI (MobileIndustry Processor Interface), SLVS-EC (Scalable Low Voltage SignalingEmbedded Clock), or another standard may be used. It is to be noted thatthe specific configuration of the interface 523 is arbitrarily defined.For example, not only a component for controlling inputs and outputs,but also transmission paths such as a bus and a cable may be included inthe interface 523.

In the stacked image sensor 510, the multilayer structure of thesemiconductor substrates is formed in the module, as described above,whereby larger circuits can be mounted without involving an increase inthe sizes of the semiconductor substrates. That is, in the stacked imagesensor 510, larger circuits can be mounted while an increase in the costis restrained.

FIG. 35 depicts an example of the configuration of the circuits formedon the respective semiconductor substrates. For convenience ofexplanation the semiconductor substrates 511 to 513 are arranged on thesame plane in FIG. 35 . However, in actual, the semiconductor substrates511 to 513 are stacked, as depicted in FIG. 34 .

A light receiving section 541, an A/D conversion section 542, etc. areformed on the top semiconductor substrate 511. The light receivingsection 541 includes a plurality of unit pixels each having aphotoelectric conversion element such as a diode, and performs, for eachof unit pixels, photoelectric conversion of incident light, generateselectric signals (pixel signals) of charges which correspond to theincident light, and outputs the signals to the A/D conversion section542.

The A/D conversion section 542 generates pixel data which is digitaldata, by executing A/D conversion of the pixel signals supplied from thelight receiving section 541. The A/D conversion section 542 supplies, asimage data, a set of the generated pixel data on the unit pixels to thesemiconductor substrate 512 via the bus 521.

An image processing section 551 which is a logic circuit that executesimage processing etc. is formed on the middle semiconductor substrate512. Upon acquiring the image data supplied from the semiconductorsubstrate 511 via the bus 521, the image processing section 551 executesprescribed image processing on the image data. The details of the imageprocessing are arbitrarily defined. For example, the image processingmay include correcting defect pixels, detecting a phase difference forautofocusing, adding pixels, a digital gain, noise reduction, and thelike. Other processes may be included.

A DRAM (Dynamic Random Access Memory) 561 is formed on the bottomsemiconductor substrate 513. The DRAM 561 is capable of storing dataetc. supplied from the semiconductor substrate 512 (image processingsection 551) via the bus 522. Further, the DRAM 561 is capable of, inresponse to a request from the semiconductor substrate 512 (imageprocessing section 551) or the like, reading out stored data etc. andsuppling the data etc. to the semiconductor substrate 512 via the bus522. That is, with use of the DRAM 561, the image processing section 551can execute image processing of temporarily holding image data that isbeing processed, for example. For example, images are captured at ahigh-speed frame rate, the captured images in the frames are stored intothe DRAM 561, and the images are read out at a low-speed frame rate andoutputted, whereby what is called slow motion imaging can be performed.

With use of the DRAM 561 thus configured, the image processing section551 encodes (compresses) image data, records encoded data thus generatedinto the DRAM 561, and generates image data (decoded image data) byreading out the encoded data from the DRAM 561 and decoding the encodeddata. For example, the image processing section 551 includes an encodingsection 551A and a decoding section 551B. The encoding section 551Aencodes image data, supplies the encoded data thus generated to the DRAM561 to record the encoded data. The decoding section 551B generatesimage data (decoded image data) by decoding encoded data read out fromthe DRAM 561. When image data is recorded as encoded data (compresseddata) into the DRAM 561 in this manner, the amount of data stored in theDRAM 561 can be reduced. Accordingly, the storage region in the DRAM 561and the band use efficiency of the bus 522 can be improved.Consequently, an increase in the capacity of the DRAM 561 and anincrease in the band width of the bus 522 can be restrained so that anincrease in the production cost can be restrained.

When the aforementioned encoding-side structure 102 (for example, FIG. 5, FIG. 14 , FIG. 23 , or FIG. 30 ) of the image processing system 100 isadopted as the encoding section 551A and the decoding-side structure 103(for example, FIG. 5 , FIG. 14 , FIG. 23 , or FIG. 30 ) is adopted asthe decoding section 551B, the aforementioned effects of the imageprocessing system 100 (for example, FIG. 5 , FIG. 14 , FIG. 23 , or FIG.30 ) can be provided. For the specific configuration of theencoding-side structure 102 and the specific configuration of thedecoding-side structure 103, any of those based on the methods #1 to #4may be adopted.

As a result of this, even in a case where, for example, high-digitalgain imaging of multiplying image data by a digital gain by means of theimage processing section 551 is executed, an average signal valuedeviation which is generated by encoding and decoding of an amplifiedsignal group can be restrained. Consequently, for example, deteriorationin the accuracy of data (as a measurement result) can be restrained. Inaddition, for example, deterioration in the subjective image quality ofa decoded image can be restrained. Moreover, deterioration in theencoding efficiency can be restrained so that deterioration in thesubjective image quality of a decoded image can be restrained.

In addition, an image processing section 571 that is a logic circuit forexecuting image processing etc. is formed on the circuit substrate 530.Upon acquiring image data supplied from the semiconductor substrate 512(image processing section 551) of the stacked image sensor 510 via theinterface 523, the image processing section 571 executes prescribedimage processing on the image data. The details of the image processingare arbitrarily defined.

That is, the image processing section 551 is capable of supplying dataetc. to the image processing section 571 via the interface 523(outputting data etc. to the outside of the stacked image sensor 510).In such a case, the image processing section 551 encodes (compresses)image data and outputs the encoded data. For example, the imageprocessing section 551 includes an encoding section 551C, and the imageprocessing section 571 includes a decoding section 571A. The encodingsection 551C encodes image data and outputs the encoded data via theinterface 523. The decoding section 571A generates image data (decodedimage data) by decoding the encoded data supplied via the interface 523.The image processing section 571 executes image processing on thedecoded image data thus generated.

When encoded data (compressed data) is transmitted via the interface 523in this manner, the amount of transmission data can be reduced.Accordingly, the efficiency of using the band of the interface 523 canbe improved. That is, an increase in the band width of the interface 523can be restrained so that an increase in the production cost can berestrained.

When the aforementioned encoding-side structure 102 (for example, FIG.13 , FIG. 22 , FIG. 29 , or FIG. 33 ) of the image processing system 100is adopted as the encoding section 551C and the decoding-side structure103 (for example, FIG. 13 , FIG. 22 , FIG. 29 , or FIG. 33 ) is adoptedas the decoding section 571A, the aforementioned effects of the imageprocessing system 100 (for example, FIG. 13 , FIG. 22 , FIG. 29 , orFIG. 33 ) can be provided. For the specific configuration of theencoding-side structure 102 and the specific configuration of thedecoding-side structure 103, any of those based on the methods #1 to #4may be adopted.

As a result of this, even in a case where, for example, high-digitalgain imaging of multiplying image data by a digital gain by means of theimage processing section 551 is executed, an average signal valuedeviation which is generated through encoding and decoding of anamplified signal group can be restrained. Consequently, for example,deterioration in the accuracy of data (as a measurement result) can berestrained. In addition, for example, deterioration in the subjectiveimage quality of a decoded image can be restrained. Moreover,deterioration in the encoding efficiency can be restrained so thatdeterioration in the subjective image quality of a decoded image can berestrained.

An example of the flow of an imaging process of capturing an image byusing the stacked image sensor 510 will be explained with reference to aflowchart in FIG. 36 .

When the imaging process is started, the light receiving section 541captures an image of a subject, and photoelectrically converts incidentlight, at step S501.

At step S502, the A/D conversion section 542 executes A/D conversion ofan electric signal generated at step S501, thereby generates image datawhich is digital data.

At step S503, the image processing section 551 acquires the image datagenerated at step S502 via the bus 521, and executes prescribed imageprocessing on the image data, that is, multiplies the image data by adigital gain, for example.

At step S504, the encoding section 551A encodes image data to berecorded into the DRAM 561. Encoding in this case is executed in a wayexplained previously in any of the first to fourth embodiments (that is,by any one of the methods #1 to #4).

At step S505, the DRAM 561 acquires the encoded data generated at stepS505 via the bus 522, and records the encoded data.

At step S506, the DRAM 561 reads out, from among encoded data recordedtherein, encoded data corresponding to a request, and supplies theread-out data to the image processing section 551 via the bus 522.

At step S507, the decoding section 551B decodes the encoded data.Decoding in this case is executed in a way explained previously in anyof the first to fourth embodiments (that is, by any one of the methods#1 to #4). The image processing section 551 may execute prescribed imageprocessing on the image data (decoded image data) generated by decoding.

At step S508, the encoding section 551C encodes image data to beoutputted to the outside of the stacked image sensor 510. Encoding inthis case is executed in a way explained previously in any of the firstto fourth embodiments (that is, by any one of the methods #1 to #4).

At step S509, the encoding section 551C supplies the encoded datagenerated at step S508, to the outside of the stacked image sensor 510(e.g., to the image processing section 571 of the circuit substrate 530)via the interface 523.

The decoding section 571A of the image processing section 571 generatesimage data (decoded image data) by decoding the supplied encoded data.Decoding in this case is executed in a way explained previously in anyof the first to fourth embodiments (that is, by any one of the methods#1 to #4). The image processing section 571 executes prescribed imageprocessing on the generated image data.

When step S509 is completed, the imaging process is ended.

By executing the imaging process in the manner described so far, thestacked image sensor 510 can restrain an average signal value deviationwhich is generated through encoding and decoding of an amplified signalgroup. That is, for example, while deterioration in the subjective imagequality of a captured image obtained by high-digital gain imaging isrestrained, an increase in the production cost of the stacked imagesensor 510 can be restrained.

It is to be noted that the configuration of the stacked image sensor 510is arbitrarily defined, and thus, is not limited to the aforementionedexample. For example, the semiconductor substrates 511 to 513 in thestacked image sensor 510 do not need to be stacked. For example, thesemiconductor substrates 511 to 513 may be arranged side by side so asto have a plane shape. In addition, the circuit configuration formed oneach of the semiconductor substrates 511 to 513 is not limited to theaforementioned example.

Moreover, the number of semiconductor substrates in the stacked imagesensor 510 may be two or less, or may be four or more. For example, theimage processing section 551 and the DRAM 561 (including the bus 522)may be formed on one semiconductor substrate.

8. Sixth Embodiment

<Application Example: Imaging Device>

FIG. 37 is a block diagram depicting a main configuration example of animaging device to which the present technology is applied. An imagingdevice 600 depicted in FIG. 37 is a device that captures an image of asubject and outputs the image of the subject in the form of an electricsignal.

As depicted in FIG. 37 , the imaging device 600 includes a controlsection 601 and a bus 610. Further, the imaging device 600 includes anoptical section 611, an image sensor 612, an image processing section613, a codec processing section 614, a display section 615, a recordingsection 616, and a communication section 617. Moreover, the imagingdevice 600 includes an input section 621, an output section 622, and adrive 625.

The control section 601 is connected, via the bus 610, with the opticalsection 611 to the communication section 617, the input section 621, theoutput section 622, and the drive 625. By controlling the operations ofthese sections, the control section 601 controls the entire process inthe imaging device 600.

Light (incident light) from a subject enters the image sensor 612 viathe optical section 611. The optical section 611 includes an arbitraryoptical element, and is driven, under control of the control section601, to exert a certain optical influence on the incident light. Forexample, the optical section 611 includes a lens that adjusts the focalpoint with respect to a subject, and collects light from the focusedposition, an aperture that executes exposure adjustment, a shutter thatcontrols an imaging timing, and the like.

The image sensor 612 receives the incident light, and executesphotoelectric conversion thereon to generate image data. The imagesensor 612 supplies the image data to the image processing section 613.

The image processing section 613 executes prescribed image processing onthe supplied image data. The details of the image processing arearbitrarily defined. For example, the image processing section 613 maybe configured to execute demosaic processing, advanced correction ofdefective pixels, or the like, on the supplied image data (RAW data).The image processing section 613 supplies the image data havingundergone the image processing, to the codec processing section 614.

The codec processing section 614 encodes image data, and decodes encodeddata, as appropriate. For example, the codec processing section 614encodes image data supplied from the image processing section 613, by aprescribed encoding method that is suitable for encoding the image data.An encoding method in this case is arbitrarily defined. For example, anadvanced compression method such as JPEG (Joint Photographic ExpertsGroup), JPEG2000, MPEG (Moving Picture Experts Group), AVC (AdvancedVideo Coding), or HEVC (High Efficiency Video Coding) can be adopted.

The codec processing section 614 is capable of generating and supplyingencoded data to the recording section 616, for example, to record theencoded data, or generating and supplying encoded data to thecommunication section 617 to output the encoded data to the outside ofthe imaging device 600. It is to be noted that the codec processingsection 614 may supply image data supplied from the image processingsection 613, for example, to the display section 615 without encodingthe image data, and causes the display section 615 to display the image.

In addition, the codec processing section 614 is capable of reading outencoded data recorded in the recording section 616, for example. Forexample, the codec processing section 614 is capable of outputting theencoded data to the outside of the imaging device 600 via thecommunication section 617.

Moreover, the codec processing section 614 is also capable ofdecompressing image data by decoding the encoded data. For example, thecodec processing section 614 is capable of supplying the decompressedimage data to the display section 615 such that an image correspondingto the image data is displayed. Furthermore, for example, the codecprocessing section 614 is capable of encoding the decompressed imagedata by another method, and then, supplying the encoded data thusgenerated to the recording section 616 to record the encoded data, orsupplying the encoded data to the communication section 617 to outputthe encoded data to the outside of the imaging device 600.

For example, under control of the control section 601, the codecprocessing section 614 executes a necessary process by selecting any oneof the aforementioned processes, as appropriate.

The display section 615 includes an arbitrary display device such as anLCD (Liquid Crystal Display), is driven, under control of the controlsection 601, to cause the display device to display an image of imagedata supplied from the codec processing section 614.

The recording section 616 includes an arbitrary recording medium such asa hard disk or a flash memory, is driven, under control of the controlsection 601, to cause the recording medium to record encoded datasupplied from the codec processing section 614. Any type medium can beused as the recording medium. A removable medium that is attachable toand detachable from the imaging device 600 may be used. In this case,the recording section 616 includes a drive (not illustrated) that canaccess data in a removable medium when the removable medium is attachedto the drive, and of the removable medium attached to the drive. Therecording section 616 may include a plurality of the recording media, ormay include a plurality of types of the recording media.

The communication section 617 is a communication interface thatcommunicates with a device external to the imaging device 600 by aprescribed communication scheme, and is driven under control of thecontrol section 601. The communication section 617 may perform any kindof communication, which may be wired communication or may be wirelesscommunication. The communication section 617 transmits encoded datasupplied from the codec processing section 614, for example, to aseparate device.

The input section 621 includes an arbitrary input device (e.g., a jogdial (registered trademark), a key, a button, or a touch panel), anexternal input terminal, or the like, and is, under control of thecontrol section 601, driven to receive an operation input from a user orthe like, and receive a control signal, data, etc. supplied from theoutside. The input section 621 supplies the received information (theoperation input, data, etc.) to the control section 601 via the bus 610.The control section 601 executes a process regarding control of theimaging device 600 in accordance with the information.

The output section 622 includes an arbitrary output device (e.g., an LED(Light Emitting Diode), a display, or a loudspeaker), an external outputterminal, or the like, and is driven, under control of the controlsection 601, to output information (e.g., data or a control signal)supplied from the control section 601 or the like (for example, displayan image, output a sound, or output information to a separate device).

The drive 625 is driven, under control of the control section 601, todrive a removable medium 631, such as a magnetic disk, an optical disk,a magneto-optical disk, or a semiconductor memory, which is attached tothe drive 625, thereby reads out information (a program, data, etc.)recorded in the removable medium 631 and supplies the information to thecontrol section 601 or the like.

As the image sensor 612 of the imaging device 600 thus configured, thestacked image sensor 510 which has been explained in the fifthembodiment is adopted. That is, the image sensor 612 is configured toexecute encoding and decoding using the present technology. It is to benoted that, in a case encoding and decoding using the present technologyare applied to an output of image data from the image sensor 612 (thatis, in a case where the encoding section 551C is adopted), a decodingsection that is equivalent to the decoding section 571A needs to beprovided to the image processing section 613.

As a result of this, the image sensor 612 can restrain an average signalvalue deviation which is generated through encoding and decoding of anamplified signal group. That is, while restraining deterioration in thesubjective image quality of a captured image obtained by high-digitalgain imaging, for example, the imaging device 600 can restrain anincrease in the production cost of the imaging device 600.

An example of the flow of an imaging process which is executed by theimaging device 600 to capture an image of a subject, will be explainedwith reference to a flowchart in FIG. 38 .

When the imaging process is started, the image sensor 612 of the imagingdevice 600 captures an image of a subject and generates image data (alsoreferred to as captured image data) on the captured image, at step S601.It is to be noted that this step is executed in the way similar to thatin the imaging process which has been explained with reference to theflowchart in FIG. 36 . That is, the image sensor 612 executes prescribedimage processing on the captured image data, encodes the data, andoutputs the encoded data. It is to be noted that the image sensor 612executes the image processing of encoding data on the captured image,recording the encoded data into the DRAM 561, and decompressing thecaptured image data by reading out the encoded data from the DRAM 561and decoding the data, as appropriate.

The image sensor 612 executes such encoding and decoding in a way usingthe present technology. That is, the image sensor 612 executes suchencoding and decoding by any one of the aforementioned methods #1 to #4.

At step S602, the image processing section 613 acquires the encoded dataoutputted from the image sensor 612.

At step S603, the image processing section 613 decodes the encoded dataacquired at step S602. This step is executed in the way similar to thatat step S507 (FIG. 36 ). That is, this step is executed by a methodcorresponding to that at step S508 (FIG. 36 ), that is, by a method thesame as that at step S508 (FIG. 36 ) in any of the aforementionedmethods #1 to #4.

At step S604, the image processing section 613 executes prescribed imageprocessing on image data on the decoded image generated at step S603.

At step S605, the display section 615 acquires the image data via thecodec processing section 614, and displays an image corresponding to theimage data.

At step S606, the codec processing section 614 acquires the image datafrom the image processing section 613, and encodes the image data.

At step S607, the recording section 616 acquires the encoded data fromthe codec processing section 614, and records the data.

At step S608, the communication section 617 acquires the encoded datafrom the codec processing section 614, and transmits the data to theoutside of the imaging device 600 (to a separate device).

When step S609 is completed, the imaging process is ended.

By executing the imaging process in the aforementioned manner, theimaging device 600 can restrain an average signal value deviation whichis generated through encoding and decoding of an amplified signal group.That is, for example, while deterioration in the subjective imagequality of a captured image obtained by high-digital gain imaging isrestrained, an increase in the production cost of the imaging device 600can be restrained.

It is to be noted that the configuration of the imaging device 600 isarbitrarily defined, and is not limited to the aforementioned example.

As examples to which the present technology is applied, the imagingelement and the imaging device have been explained above. However, thepresent technology is applicable to any device or any system as long asthe device or system executes fixed length encoding and decoding of anamplified signal group while involving quantization such as thatdisclosed in any one of PTL 1 to 4, for example.

For example, the present technology is also applicable to an imageprocessing device that acquires image data from the outside withoutexecuting imaging and executes image processing thereon. In addition, atarget to be encoded is arbitrarily defined, and thus, does not need tobe image data. For example, an arbitrary detection signal of sounds,temperature, moisture, an acceleration, or the like, which is notregarding light, can be a target to be encoded. In addition, the presenttechnology is also applicable to a device or a system that processesimage data while considering that the image data is a set of light(brightness) detection results (detection signals), for example. Forexample, the present technology is also applicable to a device or asystem that sets a black level on the basis of a set of detectionsignals.

9. Supplementary Note

<Computer>

A series of the aforementioned processes can be executed by hardware, orcan be executed by software. In a case where the series of the processesis executed by software, a program constituting the software isinstalled into a computer. Here, examples of the computer include acomputer incorporated in dedicated-hardware, and a general-purposepersonal computer capable of executing various functions by installingvarious programs.

In the case where the series of the processes is executed by software, adevice or system (e.g., the image processing system 100, the stackedimage sensor 510, or the imaging device 600) is only required to have aconfiguration as a computer capable of executing the software. Forexample, in the imaging device 600, the control section 601 (FIG. 37 )is only required to execute the series of the aforementioned processesby software by reading out a program from the recording section 616 orthe like and executing the program.

In the case where the series of the aforementioned processes is executedby software, the program, etc. constituting the software may beinstalled from a recording medium, for example. For example, in theimaging device 600, the recording medium may include the removablemedium 631 which is provided, separately from the device main body, inorder to distribute the program etc. to users, and in which the programetc. is recorded. For example, the control section 601 may read out theprogram stored in the removable medium 631 attached to the drive 625,and install the program into the recording section 616 or the like.

In addition, the program may be provided via a wired/wirelesstransmission medium such as a local area network, the internet, or adigital satellite broadcast. For example, in the imaging device 600, thecontrol section 601 may be configured to control the communicationsection 617 to receive the program provided via the transmission medium,and to install the program into the recording section 616 or the like.

Alternatively, the program may be installed in advance. For example, inthe imaging device 600, the program may be installed in advance in therecording section 616, a ROM included in the control section 601, or thelike.

<Application Target of Present Technology>

The present technology is applicable to an arbitrary image encoding anddecoding method. That is, as long as no inconsistency with the presenttechnology described so far is caused, specifications of the processesregarding image encoding and decoding are arbitrarily defined. Thespecifications are not limited to any of the aforementioned examples.

In addition, the case where the present technology is applied to animaging device has been explained above, but the present technology isapplicable to not only imaging devices, but also arbitrary devices(electronic devices). For example, the present technology is alsoapplied to an image processing device or the like for executing imageprocessing on a captured image obtained by high-digital gain imagingexecuted by means of another device.

In addition, the present technology can be implemented by any structurethat is mounted in an arbitrary device or a device constituting thesystem, such as a processor (e.g., a video processor) serving as asystem LSI (Large Scale Integration) or the like, a module (e.g., avideo module) using a plurality of processors etc., a unit (e.g., avideo unit) using a plurality of modules, a set (e.g., a video set)obtained by adding other functions to the unit (that is, the structuremeans a part of the device).

Moreover, the present technology is also applicable to a network systemincluding a plurality of devices. For example, the present technology isapplicable to a cloud service for providing image (video)-relatedservices to arbitrary terminals such as computers, AV (Audio Visual)devices, mobile information processing terminals, or IoT (Internet ofThings) devices.

It is to be noted that a system, a device, a processing section, etc. towhich the present technology is applied, can be used for an arbitraryfield pertaining to traffics, medicine, security, agriculture,stockbreeding, mining, cosmetic, industry, home electric appliances,weather, or nature monitoring. Further, an application thereof is alsoarbitrarily defined.

For example, the present technology is applicable to a system or adevice for providing viewing content etc. In addition, for example, thepresent technology is applicable to a system and a device for trafficuse such as monitoring of a traffic condition or control of automaticdriving. Moreover, for example, the present technology is applicable toa system or a device for security use. Furthermore, for example, thepresent technology is applicable to a system and a device for automaticcontrol of machines etc. Moreover, for example, the present technologyis applicable to a system and a device for agricultural or stockbreedinguse. Moreover, for example, the present technology is applicable to asystem and a device for monitoring the states of nature such asvolcanos, forests, or oceans and wildlife, etc. Moreover, for example,the present technology is applicable to a system and a device for sportsuse.

Others

In the present description, “flag” refers to information fordiscriminating a plurality of states from one another. The informationencompasses not only information which is used to discriminating twostates of true (1) and false (0) from each other, but also informationfor discriminating three or more states from one another. Therefore, thenumbers that can be taken by the “flag” may be two values which are I/O,for example, or may be three or more values. That is, the number of bitsconstituting the “flag” is arbitrarily defined, and thus, may be 1 bitor a plurality of bits. In addition, regarding identificationinformation (including a flag), it can be assumed that theidentification information is contained into a bitstream, and also,information regarding the difference of the identification informationfrom certain reference information is contained into a bitstream. Thus,the terms “flag” and “identification information” each encompass notonly information thereof, but also information regarding the differencefrom reference information.

In addition, various types of information (meta data etc.) concerningencoded data (a bitstream) can be transmitted or recorded in any form aslong as the information is associated with the encoded data. Here, theterm “associate” means, for example, enabling use of one data set whenprocessing the other data set (enabling establishment of a linktherebetween). That is, data sets that are associated with each othermay be integrated into a single data set, or may be formed as separatedata sets. For example, information associated with encoded data (image)may be transmitted over a transmission path different from that for theencoded data (image). Further, for example, information associated withencoded data (image) may be recorded into a recording medium differentfrom a recording medium in which the encoded data (image) is recorded(or in a different record area of the same recording medium). It is tobe noted that the “association” may be performed for the entirety ofdata but for a part of the data. For example, an image and informationcorresponding to the image may be associated with each other byarbitrarily defined unit of, for example, a plurality of frames, oneframe, or a part of a frame.

It is to be noted that, in the present description, the terms“synthesize,” “multiplex,” “add,” “integrate,” “include,” “store,”“place into,” “put into,” “insert,” etc. each means gathering aplurality of things together, that is, for example, gathering encodeddata and meta data into one data set, and thus, means one method for theaforementioned “association.”

Furthermore, the embodiments of the present technology are not limitedto the aforementioned embodiments, and various modifications can be madewithin the gist of the present technology.

In addition, the present technology can be implemented by any componentthat constitutes a device or system, such as a processor serving as asystem LSI (Large Scale Integration) or the like, a module using aplurality of processors etc., a unit using a plurality of modules, a setobtained by adding other functions to the unit (that is, the componentmeans a part of the device).

It is to be noted that, in the present description, a system refers to aset of a plurality of constituent elements (devices, modules(components), etc.). Whether or not the constituent elements are allincluded in the same casing does not matter. Therefore, both a set of aplurality of devices that are housed in different casings and areconnected to one another over a network, and a single device having aplurality of modules housed in a single casing are referred to assystem.

In addition, for example, the configuration explained above as a singledevice (or processing section) may be divided into a plurality ofdevices (or processing sections). Conversely, the configurationsexplained as a plurality of devices (or processing sections) may beformed into a single device (or processing section). Also, aconfiguration not described above may be added to the configurations ofthe devices (or processing sections). Moreover, as long as theconfiguration or operation in the entire system is substantially thesame, a part of a certain device (or processing section) may be includedin another device (or processing section).

In addition, for example, the present technology can have aconfiguration of cloud computing in which one function is shared andjointly processed by a plurality of devices over a network.

In addition, for example, the aforementioned program can be executed byan arbitrary device. In this case, it is sufficient that the device hasnecessary functions (functional blocks etc.) and is capable of acquiringnecessary information.

In addition, for example, the steps of the flowcharts explained abovemay be executed by one device, or may be jointly executed by a pluralityof devices. Moreover, in a case where a plurality of processes isincluded in one step, the one step may be executed by one device, or maybe jointly executed by a plurality of devices. In other words, aplurality of processes included in one step may be executed like aplurality of steps. Conversely, a plurality of the steps in the aboveexplanation may be collectively executed like one step.

It is to be noted that the program which is executed by a computer maybe a program for executing the processes in accordance with thetime-series order explained in the present description, or may be aprogram for executing the processes separately at necessary timings,such as each time a call is made. That is, as long as no inconsistencyis produced, the steps may be executed in accordance with an order thatis different from the aforementioned one. Moreover, steps written in theprogram may be executed in parallel with processes of another program,or may be executed in combination with processes of another program.

It is to be noted that a plurality of exemplifications of the presenttechnology explained in the present description can be implementedindependently as long as no inconsistency is produced. A plurality ofarbitrarily defined exemplifications of the present technology can beimplemented in combination. For example, a part or the whole of thepresent technology explained in any one of the embodiments can beimplemented in combination with a part or the whole of the presenttechnology explained in another embodiment. In addition, an arbitrarilydefined part or the whole of the present technology can be implementedin combination with another technology that is has not been describedabove.

It is to be noted that the present technology also may have thefollowing configurations.

(1)

An image processing device including:

an adaptive processing section that executes adaptive image processingof an image in which signal amplification has been executed; and

an encoding section that executes simple encoding of the image havingundergone the adaptive image processing executed by the adaptiveprocessing section.

(2)

The image processing device according to (1), in which

the adaptive processing section executes the imaging processing ofadding, to each pixel value of the image, an offset value that israndomly set within a value range that depends on a gain value of thesignal amplification executed on the image, and

the encoding section executes simple encoding of the image in which theoffset value has been added to each pixel value by the adaptiveprocessing section.

(3)

The image processing device according to (2), in which

the adaptive processing section adds, as the offset value, a pseudorandom number corrected to fall within the value range that depends onthe gain value, to each pixel value of the image.

(4)

The image processing device according to any one of (1) to (3), in which

the adaptive processing section executes the image processing ofsubtracting, from each pixel value of the image, an offset value that isbased on an average pixel value of the image and a quantization value ofsimple encoding to be executed by the encoding section, and

the encoding section executes simple encoding of the image in which theoffset value has been subtracted from each pixel value by the adaptiveprocessing section.

(5)

The image processing device according to (4), in which

the average pixel value includes an average pixel value of an image of aframe prior to a current frame which is a process target.

(6)

The image processing device according to (5), in which

the quantization value includes a value that depends on a compressionrate of the simple encoding.

(7)

The image processing device according to (5) or (6), in which

the quantization value is an average of quantization values, forrespective pixels, of the simple encoding of the image of a frame priorto a current frame which is a process target.

(8)

The image processing device according to any one of (4) to (7), in which

for each color, the adaptive processing section subtracts the offsetvalue from each pixel value of the image.

(9)

The image processing device according to (4), further including:

a decoding section that executes simple decoding of encoded datagenerated by the encoding section; and

an offset adding section that adds, to each pixel value of a decodedimage generated by the decoding section, an offset value that is basedon an average pixel value of the image and a quantization value of thesimple encoding.

(10)

The image processing device according to any one of (1) to (9), in which

the adaptive processing section executes the image processing of settinga range of a quantization value of simple encoding to be executed by theencoding section, and

the encoding section executes simple encoding of the image on a basis ofthe range of a quantization value set by the adaptive processingsection, and generates encoded data including information regarding therange of a quantization value.

(11)

The image processing device according to (10), in which

the adaptive processing section sets the range of a quantization valueaccording to a gain value of the signal amplification executed on theimage.

(12)

The image processing device according to (10), further including:

a decoding section that executes simple decoding of the encoded datagenerated by the encoding section, on the basis of the informationregarding the range of a quantization value included in the encodeddata.

(13)

The image processing device according to any one of (1) to (12), inwhich

the adaptive processing section executes the image processing ofdividing each pixel value of the image by a gain value of the signalamplification executed on the image, and

the encoding section executes simple encoding of the image in which eachpixel value has been divided by the gain value by the adaptiveprocessing section.

(14)

The image processing device according to (13), further including:

a decoding section that decodes encoded data generated by the encodingsection; and

a gain value multiplication section that multiplies, by the gain value,each pixel value of a decoded image generated by the decoding section.

(15)

The image processing device according to any one of (1) to (14), furtherincluding:

an amplification section that executes signal amplification on theimage, in which

the adaptive processing section executes the adaptive image processingof the image in which signal amplification has been executed by theamplification section.

(16)

The image processing device according to any one of (1) to (15), furtherincluding:

a gain value setting section that sets a gain value of the signalamplification executed on the image.

(17)

The image processing device according to any one of (1) to (16), furtherincluding:

a recording section that records encoded data generated by the encodingsection.

(18)

An image processing method including:

executing adaptive image processing of an image in which signalamplification has been executed; and

executing simple encoding of the image having undergone the adaptiveimage processing.

(19)

An imaging element including:

an imaging section that captures an image of a subject;

an adaptive processing section that executes adaptive image processingof the captured image which has been generated by the imaging sectionand in which signal amplification has been executed; and

an encoding section that executes simple encoding of the captured imagehaving undergone the adaptive image processing executed by the adaptiveprocessing section.

(20)

An imaging device including:

an imaging element including

-   -   an imaging section that captures an image of a subject,    -   an adaptive processing section that executes adaptive image        processing of the captured image which has been generated by the        imaging section and in which signal amplification has been        executed, and    -   an encoding section that generates encoded data by executing        simple encoding of the captured image having undergone the        adaptive image processing executed by the adaptive processing        section; and

a decoding section that executes simple decoding of the encoded datagenerated by the encoding section.

REFERENCE SIGNS LIST

100 Image processing system, 101 Control section, 102 Encoding-sidestructure, 103 Decoding-side structure, 111 Amplification section, 112Random offset adding section, 113 Encoding section, 121 Decodingsection, 141 Pseudo random number generation section, 142 Value rangelimiting section, 143 Computing section, 144 Clipping section, 171Transmission section, 172 Reception section, 211 Subtraction offsetsetting section, 212 Computing section, 213 Clipping section, 221Addition offset setting section, 222 Computing section, 223 Clippingsection, 231 Average value measuring section, 232 Offset value selectionsection, 233 Offset value supply section, 251 Compression section, 252Average value measuring section, 311 Quantization value range settingsection, 411 Computing section, 421 Computing section, 510 Stacked imagesensor, 511 to 513 Semiconductor substrate, 521, 522 Bus, 523 Interface,530 Circuit substrate, 541 Light receiving section, 542 A/D conversionsection, 551 Image processing section, 561 DRAM, 571 Image processingsection, 600 Imaging device, 601 Control section, 610 Bus, 611 Opticalsection, 612 Image sensor, 613 Image processing section, 614 Codecprocessing section, 615 Display section, 616 Recording section, 617Communication section, 621 Input section, 622 Output section, 625 Drive

The invention claimed is:
 1. An image processing device, comprising: anadaptive processing section configured to execute adaptive imageprocessing of an image in which signal amplification has been executed,wherein the adaptive image processing is executed to set a range of aquantization value of simple encoding; and an encoding sectionconfigured to: execute, based on the range of the quantization value,the simple encoding of the image having undergone the adaptive imageprocessing, and generate, based on the execution of the simple encoding,encoded data that includes information regarding the range of thequantization value.
 2. The image processing device according to claim 1,wherein the adaptive processing section is further configured to executethe adaptive imaging processing to add, to each pixel value of theimage, an offset value that is randomly set within a value range thatdepends on a gain value of the signal amplification executed on theimage, and the encoding section is further configured to execute thesimple encoding of the image in which the offset value has been added toeach pixel value.
 3. The image processing device according to claim 2,wherein the adaptive processing section is further configured to add, asthe offset value, a pseudo random number corrected to fall within thevalue range that depends on the gain value, to each pixel value of theimage.
 4. The image processing device according to claim 1, wherein theadaptive processing section is further configured to execute theadaptive image processing to subtract, from each pixel value of theimage, an offset value that is based on an average pixel value of theimage and the quantization value, and the encoding section is furtherconfigured to execute the simple encoding of the image in which theoffset value has been subtracted from each pixel value.
 5. The imageprocessing device according to claim 4, wherein the average pixel valueincludes an average pixel value of an image of a frame prior to acurrent frame which is a process target.
 6. The image processing deviceaccording to claim 5, wherein the quantization value includes a valuethat depends on a compression rate of the simple encoding.
 7. The imageprocessing device according to claim 5, wherein the quantization valueis an average of quantization values, for respective pixels, of thesimple encoding of the image of the frame prior to the current framewhich is the process target.
 8. The image processing device according toclaim 4, wherein for each color, the adaptive processing section isfurther configured to subtract the offset value from each pixel value ofthe image.
 9. The image processing device according to claim 4, furthercomprising: a decoding section configured to: execute simple decoding ofthe generated encoded data, and generate a decoded image based on theexecution of the simple decoding; and an offset adding sectionconfigured to add, to each pixel value of the decoded image, the offsetvalue that is based on the average pixel value of the image and thequantization value.
 10. The image processing device according to claim1, wherein the adaptive processing section is further configured to setthe range of the quantization value according to a gain value of thesignal amplification executed on the image.
 11. The image processingdevice according to claim 1, further comprising a decoding sectionconfigured to execute simple decoding of the generated encoded databased on the information regarding the range of the quantization valueincluded in the generated encoded data.
 12. The image processing deviceaccording to claim 1, wherein the adaptive processing section is furtherconfigured to execute the adaptive image processing to divide each pixelvalue of the image by a gain value of the signal amplification executedon the image, and the encoding section is further configured to executethe simple encoding of the image in which each pixel value has beendivided by the gain value.
 13. The image processing device according toclaim 12, further comprising: a decoding section configured to: executesimple decoding of the generated encoded data, and generate a decodedimage based on the execution of the simple decoding; and a gain valuemultiplication section configured to multiply, by the gain value, eachpixel value of the decoded image.
 14. The image processing deviceaccording to claim 1, further comprising an amplification sectionconfigured to execute the signal amplification on the image.
 15. Theimage processing device according to claim 1, further comprising a gainvalue setting section configured to set a gain value of the signalamplification executed on the image.
 16. The image processing deviceaccording to claim 1, further comprising a recording section configuredto record the generated encoded data.
 17. An image processing method,comprising: executing adaptive image processing of an image in whichsignal amplification has been executed, wherein the adaptive imageprocessing is executed for setting a range of a quantization value ofsimple encoding; executing, based on the range of the quantizationvalue, the simple encoding of the image having undergone the adaptiveimage processing; and generating, based on the execution of the simpleencoding, encoded data that includes information regarding the range ofthe quantization value.
 18. An imaging element, comprising: an imagingsection configured to capture an image of a subject; an adaptiveprocessing section configured to execute adaptive image processing ofthe captured image in which signal amplification has been executed,wherein the adaptive image processing is executed to set a range of aquantization value of simple encoding; and an encoding sectionconfigured to: execute, based on the range of the quantization value,the simple encoding of the captured image having undergone the adaptiveimage processing, and generate, based on the execution of the simpleencoding, encoded data that includes information regarding the range ofthe quantization value.
 19. An imaging device, comprising: an imagingelement that includes: an imaging section configured to capture an imageof a subject, an adaptive processing section configured to executeadaptive image processing of the captured image in which signalamplification has been executed, wherein the adaptive image processingis executed to set a range of a quantization value of simple encoding,and an encoding section configured to: execute, based on the range ofthe quantization value, the simple encoding of the captured image havingundergone the adaptive image processing, and generate, based on theexecution of the simple encoding, encoded data that includes informationregarding the range of the quantization value; and a decoding sectionconfigured to execute simple decoding of the generated encoded data.